[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:43:52 dist-test-slave-tvl9 krb5kdc[15676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:43:52 dist-test-slave-tvl9 krb5kdc[15676](info): set up 2 sockets
Jun 20 18:43:52 dist-test-slave-tvl9 krb5kdc[15676](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260620 18:43:53.955197 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:43:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:43:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:43:54.078004 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:43621
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:43621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:54.163025 15689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:54.163255 15689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:54.163280 15689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:54.164567 15689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:43:54.164613 15689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:54.164626 15689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:54.164640 15689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:43:54.164652 15689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:43:54.166121 15689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:43621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:43621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:54.166481 15689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:54.166692 15689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:43:54.172086 15695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:54.172051 15694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:54.172046 15697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:54.172298 15689 server_base.cc:1034] running on GCE node
I20260620 18:43:54.172650 15689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:54.172986 15689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:54.174186 15689 hybrid_clock.cc:648] HybridClock initialized: now 1781981034174165 us; error 41 us; skew 500 ppm
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:54.180310 15689 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:43:54.180846 15689 webserver.cc:460] Webserver started at http://127.15.77.126:35173/ using document root <none> and password file <none>
I20260620 18:43:54.181129 15689 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:54.181203 15689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:54.181356 15689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:54.182571 15689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "236ed9711a954d4b8b87c9aed3052cc9"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "e4bb464c5c3d06aa0101663e52b9c2e5"
server_key_iv: "d7beaab1f23885ec723ee95dc245b0b8"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.183025 15689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "236ed9711a954d4b8b87c9aed3052cc9"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "e4bb464c5c3d06aa0101663e52b9c2e5"
server_key_iv: "d7beaab1f23885ec723ee95dc245b0b8"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.184790 15689 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260620 18:43:54.185791 15704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:54.186030 15689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:43:54.186164 15689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "236ed9711a954d4b8b87c9aed3052cc9"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "e4bb464c5c3d06aa0101663e52b9c2e5"
server_key_iv: "d7beaab1f23885ec723ee95dc245b0b8"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.186264 15689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:54.203965 15689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:54.206379 15689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:54.206550 15689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:54.210760 15689 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:43621
I20260620 18:43:54.210799 15756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:43621 every 8 connection(s)
I20260620 18:43:54.211155 15689 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:43:54.212131 15757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:54.213313 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 15689
I20260620 18:43:54.213393 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:43:54.214051 15669 external_mini_cluster.cc:1428] Setting key ce916c6676172c802b2b4c147893e8cf
I20260620 18:43:54.214759 15757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9: Bootstrap starting.
I20260620 18:43:54.215715 15757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:54.216113 15757 log.cc:824] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9: Log is configured to *not* fsync() on all Append() calls
I20260620 18:43:54.216823 15757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9: No bootstrap required, opened a new log
I20260620 18:43:54.218470 15757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236ed9711a954d4b8b87c9aed3052cc9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43621 } }
I20260620 18:43:54.218631 15757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:54.218711 15757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236ed9711a954d4b8b87c9aed3052cc9, State: Initialized, Role: FOLLOWER
I20260620 18:43:54.218932 15757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236ed9711a954d4b8b87c9aed3052cc9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43621 } }
I20260620 18:43:54.219038 15757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:43:54.219089 15757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:43:54.219139 15757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:54.220045 15757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236ed9711a954d4b8b87c9aed3052cc9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43621 } }
I20260620 18:43:54.220172 15757 leader_election.cc:302] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 236ed9711a954d4b8b87c9aed3052cc9; no voters:
I20260620 18:43:54.220381 15757 leader_election.cc:288] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:43:54.220407 15762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:43:54.220543 15762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [term 1 LEADER]: Becoming Leader. State: Replica: 236ed9711a954d4b8b87c9aed3052cc9, State: Running, Role: LEADER
I20260620 18:43:54.220819 15757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:43:54.220788 15762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236ed9711a954d4b8b87c9aed3052cc9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43621 } }
I20260620 18:43:54.221313 15764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 236ed9711a954d4b8b87c9aed3052cc9. Latest consensus state: current_term: 1 leader_uuid: "236ed9711a954d4b8b87c9aed3052cc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236ed9711a954d4b8b87c9aed3052cc9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43621 } } }
I20260620 18:43:54.221395 15764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [sys.catalog]: This master's current role is: LEADER
I20260620 18:43:54.221705 15763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "236ed9711a954d4b8b87c9aed3052cc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236ed9711a954d4b8b87c9aed3052cc9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43621 } } }
I20260620 18:43:54.221769 15763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9 [sys.catalog]: This master's current role is: LEADER
W20260620 18:43:54.222803 15777 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:43:54.222872 15777 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:43:54.222934 15770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:43:54.223114 15770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:43:54.224924 15770 catalog_manager.cc:1347] Generated new cluster ID: e35487e3b7934b489849a73421c186b2
I20260620 18:43:54.224978 15770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:54.233496 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:54.214970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55212 (local address 127.15.77.126:43621)
0620 18:43:54.215215 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:54.215226 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:54.215470 (+ 244us) server_negotiation.cc:407] Connection header received
0620 18:43:54.218448 (+ 2978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:54.218454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:54.218502 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:54.218655 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:54.220385 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.221814 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.223462 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.224040 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.224165 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:54.228541 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:54.228545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:54.228545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:54.228557 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:54.231193 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.231764 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.231767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.231767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.231882 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.232178 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.232178 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.232179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.232463 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:54.232620 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:54.232921 (+ 301us) server_negotiation.cc:299] Negotiation successful
0620 18:43:54.233063 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260620 18:43:54.239395 15770 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:43:54.239935 15770 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:43:54.246487 15770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 236ed9711a954d4b8b87c9aed3052cc9: Generated new TSK 0
I20260620 18:43:54.246744 15770 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:43:54.273773 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:43621
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:54.356989 15783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:54.357201 15783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:54.357225 15783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:54.358397 15783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:54.358440 15783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:54.358469 15783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:43:54.359900 15783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:43621
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:54.360095 15783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:54.360278 15783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:43:54.365597 15788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:54.365597 15791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:54.365629 15789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:54.365702 15783 server_base.cc:1034] running on GCE node
I20260620 18:43:54.365993 15783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:54.366238 15783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:54.367394 15783 hybrid_clock.cc:648] HybridClock initialized: now 1781981034367360 us; error 46 us; skew 500 ppm
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:54.369824 15783 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:43:54.370334 15783 webserver.cc:460] Webserver started at http://127.15.77.65:35687/ using document root <none> and password file <none>
I20260620 18:43:54.370560 15783 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:54.370604 15783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:54.370754 15783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:54.371884 15783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "1e638f2e860747359eefc8920ff1c903"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "7e6f4effda114f4c0249090f848afd89"
server_key_iv: "4c7f9e8629f78b2e806e6b49182f8eb8"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.372251 15783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "1e638f2e860747359eefc8920ff1c903"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "7e6f4effda114f4c0249090f848afd89"
server_key_iv: "4c7f9e8629f78b2e806e6b49182f8eb8"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.373557 15783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:43:54.374300 15798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:54.374495 15783 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:43:54.374591 15783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "1e638f2e860747359eefc8920ff1c903"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "7e6f4effda114f4c0249090f848afd89"
server_key_iv: "4c7f9e8629f78b2e806e6b49182f8eb8"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.374665 15783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:54.388247 15783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:54.390444 15783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:54.390602 15783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:54.390851 15783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:43:54.391176 15783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:43:54.391232 15783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:54.391268 15783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:43:54.391294 15783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:54.397279 15783 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:36313
I20260620 18:43:54.397323 15911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:36313 every 8 connection(s)
I20260620 18:43:54.397651 15783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:43:54.398699 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 15783
I20260620 18:43:54.398769 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:43:54.398938 15669 external_mini_cluster.cc:1428] Setting key 544564d5f03b656628632325aea0d7a3
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:43:54.407889 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:54.398370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:34257 (local address 127.15.77.126:43621)
0620 18:43:54.398448 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:54.398451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:54.398665 (+ 214us) server_negotiation.cc:407] Connection header received
0620 18:43:54.399404 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:54.399406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:54.399423 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:54.399504 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:54.400517 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.401032 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.402067 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.402296 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.402383 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:54.404878 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:54.404880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:54.404880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:54.404888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:54.406384 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.406867 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.406869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.406869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.406932 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.407181 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.407182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.407182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.407299 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:54.407401 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:54.407711 (+ 310us) server_negotiation.cc:299] Negotiation successful
0620 18:43:54.407816 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 18:43:54.408182 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:54.398343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:43621 (local address 127.15.77.65:34257)
0620 18:43:54.398532 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:54.398569 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:54.399279 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:54.399583 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:54.399587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:54.400218 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:54.400411 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:54.400415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.401132 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.401133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:54.401973 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:54.401976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.402396 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.402397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:54.402531 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:54.403079 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:54.403088 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:54.404774 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:54.406539 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:54.406548 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:54.406550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:54.406791 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:54.406994 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:54.406995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:54.406995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:54.407105 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:54.407410 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:54.407417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:54.407544 (+ 127us) client_negotiation.cc:769] Sending connection context
0620 18:43:54.407686 (+ 142us) client_negotiation.cc:240] Negotiation successful
0620 18:43:54.407946 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260620 18:43:54.408743 15912 heartbeater.cc:347] Connected to a master server at 127.15.77.126:43621
I20260620 18:43:54.408875 15912 heartbeater.cc:464] Registering TS with master...
I20260620 18:43:54.409030 15912 heartbeater.cc:511] Master 127.15.77.126:43621 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
I20260620 18:43:54.409668 15721 ts_manager.cc:194] Registered new tserver with Master: 1e638f2e860747359eefc8920ff1c903 (127.15.77.65:36313)
I20260620 18:43:54.410632 15721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:34257
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:43:54.426324 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:43621
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:54.509968 15917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:54.510241 15917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:54.510279 15917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:54.511533 15917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:54.511615 15917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:54.511660 15917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:43:54.513188 15917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:43621
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:54.513479 15917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:54.513743 15917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:43:54.518934 15922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:54.519017 15925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:54.519186 15923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:54.519068 15917 server_base.cc:1034] running on GCE node
I20260620 18:43:54.519508 15917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:54.519737 15917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:54.520888 15917 hybrid_clock.cc:648] HybridClock initialized: now 1781981034520867 us; error 36 us; skew 500 ppm
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:54.523595 15917 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:43:54.524091 15917 webserver.cc:460] Webserver started at http://127.15.77.66:42453/ using document root <none> and password file <none>
I20260620 18:43:54.524350 15917 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:54.524418 15917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:54.524571 15917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:54.525630 15917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "67c57e4f39dc41f9b141e4799e46eac1"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "3da620149de0e1cd7d470930829d1ef9"
server_key_iv: "a332a461c3edc7943c5e62e0bbe3c721"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.525997 15917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "67c57e4f39dc41f9b141e4799e46eac1"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "3da620149de0e1cd7d470930829d1ef9"
server_key_iv: "a332a461c3edc7943c5e62e0bbe3c721"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.527298 15917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:43:54.528146 15932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:54.528330 15917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:43:54.528429 15917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "67c57e4f39dc41f9b141e4799e46eac1"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "3da620149de0e1cd7d470930829d1ef9"
server_key_iv: "a332a461c3edc7943c5e62e0bbe3c721"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.528512 15917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:54.542146 15917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:54.544274 15917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:54.544441 15917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:54.544690 15917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:43:54.545078 15917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:43:54.545135 15917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:54.545169 15917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:43:54.545210 15917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:54.551033 15917 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:45593
I20260620 18:43:54.551079 16045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:45593 every 8 connection(s)
I20260620 18:43:54.551465 15917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:54.560658 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:54.552259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:53233 (local address 127.15.77.126:43621)
0620 18:43:54.552325 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:54.552327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:54.552529 (+ 202us) server_negotiation.cc:407] Connection header received
0620 18:43:54.553088 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:54.553090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:54.553104 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:54.553161 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:54.554122 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.554629 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.555642 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.555848 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.555939 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:54.558038 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:54.558039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:54.558039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:54.558047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:54.559245 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.559705 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.559706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.559707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.559769 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.560007 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.560008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.560009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.560132 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:54.560206 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:54.560477 (+ 271us) server_negotiation.cc:299] Negotiation successful
0620 18:43:54.560563 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 18:43:54.560843 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:54.552246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:43621 (local address 127.15.77.66:53233)
0620 18:43:54.552408 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:54.552444 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:54.552973 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:54.553232 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:54.553241 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:54.553809 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:54.554006 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:54.554010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.554724 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.554725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:54.555541 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:54.555543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.555948 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.555949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:54.556077 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:54.556611 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:54.556620 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:54.557942 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:54.559355 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:54.559361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:54.559363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:54.559612 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:54.559836 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:54.559837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:54.559838 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:54.559941 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:54.560206 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:54.560213 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:54.560345 (+ 132us) client_negotiation.cc:769] Sending connection context
0620 18:43:54.560444 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 18:43:54.560610 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260620 18:43:54.561339 16046 heartbeater.cc:347] Connected to a master server at 127.15.77.126:43621
I20260620 18:43:54.561420 16046 heartbeater.cc:464] Registering TS with master...
I20260620 18:43:54.561672 16046 heartbeater.cc:511] Master 127.15.77.126:43621 requested a full tablet report, sending...
I20260620 18:43:54.561694 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 15917
I20260620 18:43:54.561796 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:43:54.561975 15669 external_mini_cluster.cc:1428] Setting key 178c0a3eb7cacbe7576d231aa8b734d3
I20260620 18:43:54.562114 15721 ts_manager.cc:194] Registered new tserver with Master: 67c57e4f39dc41f9b141e4799e46eac1 (127.15.77.66:45593)
I20260620 18:43:54.562706 15721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:53233
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:43:54.586642 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:43621
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:54.668572 16051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:54.668772 16051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:54.668797 16051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:54.669950 16051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:54.669991 16051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:54.670018 16051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:43:54.671543 16051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:43621
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:54.671794 16051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:54.672029 16051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:43:54.677026 16051 server_base.cc:1034] running on GCE node
W20260620 18:43:54.677007 16056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:54.677007 16059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:54.677001 16057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:54.677378 16051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:54.677675 16051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:54.678833 16051 hybrid_clock.cc:648] HybridClock initialized: now 1781981034678800 us; error 46 us; skew 500 ppm
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:54.681317 16051 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:43:54.681744 16051 webserver.cc:460] Webserver started at http://127.15.77.67:45253/ using document root <none> and password file <none>
I20260620 18:43:54.681993 16051 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:54.682036 16051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:54.682196 16051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:54.683261 16051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "4b85c7c913a6489485944b9b29f898cf"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "b1568bca7f3cf8965901c2ac61f983c6"
server_key_iv: "8b9d3ee0127ca73b0d2c37488b3a837f"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.683696 16051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "4b85c7c913a6489485944b9b29f898cf"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "b1568bca7f3cf8965901c2ac61f983c6"
server_key_iv: "8b9d3ee0127ca73b0d2c37488b3a837f"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.685071 16051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:43:54.685887 16066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:54.686061 16051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:43:54.686160 16051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "4b85c7c913a6489485944b9b29f898cf"
format_stamp: "Formatted at 2026-06-20 18:43:54 on dist-test-slave-tvl9"
server_key: "b1568bca7f3cf8965901c2ac61f983c6"
server_key_iv: "8b9d3ee0127ca73b0d2c37488b3a837f"
server_key_version: "encryptionkey@0"
I20260620 18:43:54.686234 16051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:54.701673 16051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:54.703763 16051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:54.703934 16051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:54.704186 16051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:43:54.704516 16051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:43:54.704551 16051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:54.704574 16051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:43:54.704622 16051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:54.710408 16051 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:33807
I20260620 18:43:54.710505 16179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:33807 every 8 connection(s)
I20260620 18:43:54.710841 16051 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:43:54.711547 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 16051
I20260620 18:43:54.711653 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:43:54.711839 15669 external_mini_cluster.cc:1428] Setting key 9b7ca1e05516d2bc732be8864bd3a9ec
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:54.720322 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:54.711635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:39971 (local address 127.15.77.126:43621)
0620 18:43:54.711686 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:54.711689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:54.711937 (+ 248us) server_negotiation.cc:407] Connection header received
0620 18:43:54.712584 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:54.712585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:54.712598 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:54.712643 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:54.713600 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.714106 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.715129 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.715396 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.715573 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:54.717634 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:54.717636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:54.717636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:54.717644 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:54.718872 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.719294 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.719295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.719296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.719383 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.719659 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.719660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.719661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.719787 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:54.719867 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:54.720144 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 18:43:54.720228 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:43:54.720574 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:54.711642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:43621 (local address 127.15.77.67:39971)
0620 18:43:54.711790 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:54.711833 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:54.712484 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:54.712715 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:54.712719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:54.713307 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:54.713503 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:54.713507 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.714223 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.714224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:54.715026 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:54.715028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.715582 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.715584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:54.715723 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:54.716187 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:54.716195 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:54.717536 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:54.718964 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:54.718972 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:54.718974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:54.719200 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:54.719471 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:54.719472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:54.719472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:54.719589 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:54.719856 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:54.719863 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:54.720012 (+ 149us) client_negotiation.cc:769] Sending connection context
0620 18:43:54.720122 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 18:43:54.720283 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260620 18:43:54.720994 16180 heartbeater.cc:347] Connected to a master server at 127.15.77.126:43621
I20260620 18:43:54.721120 16180 heartbeater.cc:464] Registering TS with master...
I20260620 18:43:54.721338 16180 heartbeater.cc:511] Master 127.15.77.126:43621 requested a full tablet report, sending...
I20260620 18:43:54.721760 15721 ts_manager.cc:194] Registered new tserver with Master: 4b85c7c913a6489485944b9b29f898cf (127.15.77.67:33807)
I20260620 18:43:54.722143 15721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:39971
I20260620 18:43:54.724097 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:43:54.747934 16187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:54.739626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47986 (local address 127.15.77.65:36313)
0620 18:43:54.739780 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:54.739788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:54.739805 (+ 17us) server_negotiation.cc:407] Connection header received
0620 18:43:54.739833 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:54.739835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:54.739849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:54.739930 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:54.740586 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.741313 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.742148 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.742479 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.742551 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:54.745037 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:54.745046 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:54.745046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:54.745052 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:54.746571 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.746942 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.746945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.746946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.747025 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.747246 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.747247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.747247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.747574 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:54.747711 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:54.747779 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 18:43:54.747851 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
W20260620 18:43:54.748337 15826 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47986
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[15676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:43:54.772718 16187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:54.764851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47996 (local address 127.15.77.65:36313)
0620 18:43:54.764925 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:54.764927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:54.764999 (+ 72us) server_negotiation.cc:407] Connection header received
0620 18:43:54.765096 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:54.765097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:54.765111 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:54.765181 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:54.765864 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.766414 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.767232 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:54.767546 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:54.767635 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:54.770142 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:54.770144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:54.770144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:54.770151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:54.771632 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.771970 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.771971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.771971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.772022 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:54.772221 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:54.772222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:54.772222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:54.772330 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:54.772433 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:54.772575 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 18:43:54.772647 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 18:43:54.773300 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 15783
I20260620 18:43:54.778751 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 15917
I20260620 18:43:54.784173 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 16051
I20260620 18:43:54.788542 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 15689
2026-06-20T18:43:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1883 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[16194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[16194](info): set up 2 sockets
Jun 20 18:43:54 dist-test-slave-tvl9 krb5kdc[16194](info): commencing operation
krb5kdc: starting...
W20260620 18:43:55.826900 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:43:55 dist-test-slave-tvl9 krb5kdc[16194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:43:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:43:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:43:55.951514 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:32991
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:32991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:56.035610 16207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:56.035871 16207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:56.035913 16207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:56.037160 16207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:43:56.037247 16207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:56.037269 16207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:56.037290 16207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:43:56.037317 16207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:43:56.038932 16207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:32991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:32991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:56.039258 16207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:56.039546 16207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:43:56.044657 16212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:56.044665 16213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:56.044668 16215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:56.045027 16207 server_base.cc:1034] running on GCE node
I20260620 18:43:56.045300 16207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:56.045585 16207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:56.046764 16207 hybrid_clock.cc:648] HybridClock initialized: now 1781981036046750 us; error 43 us; skew 500 ppm
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:56.049654 16207 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:43:56.050226 16207 webserver.cc:460] Webserver started at http://127.15.77.126:37101/ using document root <none> and password file <none>
I20260620 18:43:56.050591 16207 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:56.050865 16207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:56.051075 16207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:56.052279 16207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "6628211d428b40b283ed47e19a0fa7d9"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "c83f8b5ac05d8a399fa58f862957563b"
server_key_iv: "d071aeff93a7760e2f05a17204a58834"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.052693 16207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "6628211d428b40b283ed47e19a0fa7d9"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "c83f8b5ac05d8a399fa58f862957563b"
server_key_iv: "d071aeff93a7760e2f05a17204a58834"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.054171 16207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:43:56.055101 16222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:56.055364 16207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:43:56.055493 16207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "6628211d428b40b283ed47e19a0fa7d9"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "c83f8b5ac05d8a399fa58f862957563b"
server_key_iv: "d071aeff93a7760e2f05a17204a58834"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.055558 16207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:56.075608 16207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:56.077922 16207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:56.078125 16207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:56.082599 16207 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:32991
I20260620 18:43:56.082636 16274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:32991 every 8 connection(s)
I20260620 18:43:56.083019 16207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:43:56.084005 16275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:56.086370 16275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9: Bootstrap starting.
I20260620 18:43:56.086742 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 16207
I20260620 18:43:56.086843 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:43:56.086994 15669 external_mini_cluster.cc:1428] Setting key e215a170ea77a013b58fa5ac037d7c11
I20260620 18:43:56.087148 16275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:56.087513 16275 log.cc:824] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9: Log is configured to *not* fsync() on all Append() calls
I20260620 18:43:56.088306 16275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9: No bootstrap required, opened a new log
I20260620 18:43:56.092538 16275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6628211d428b40b283ed47e19a0fa7d9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 32991 } }
I20260620 18:43:56.092734 16275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:56.092787 16275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6628211d428b40b283ed47e19a0fa7d9, State: Initialized, Role: FOLLOWER
I20260620 18:43:56.092954 16275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6628211d428b40b283ed47e19a0fa7d9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 32991 } }
I20260620 18:43:56.093060 16275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:43:56.093103 16275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:43:56.093152 16275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:56.093834 16275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6628211d428b40b283ed47e19a0fa7d9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 32991 } }
I20260620 18:43:56.093966 16275 leader_election.cc:302] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6628211d428b40b283ed47e19a0fa7d9; no voters:
I20260620 18:43:56.094208 16275 leader_election.cc:288] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:43:56.094264 16280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:43:56.094427 16280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [term 1 LEADER]: Becoming Leader. State: Replica: 6628211d428b40b283ed47e19a0fa7d9, State: Running, Role: LEADER
I20260620 18:43:56.094496 16275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:43:56.094535 16280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6628211d428b40b283ed47e19a0fa7d9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 32991 } }
I20260620 18:43:56.094903 16280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6628211d428b40b283ed47e19a0fa7d9. Latest consensus state: current_term: 1 leader_uuid: "6628211d428b40b283ed47e19a0fa7d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6628211d428b40b283ed47e19a0fa7d9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 32991 } } }
I20260620 18:43:56.094996 16280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [sys.catalog]: This master's current role is: LEADER
I20260620 18:43:56.094913 16282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6628211d428b40b283ed47e19a0fa7d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6628211d428b40b283ed47e19a0fa7d9" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 32991 } } }
I20260620 18:43:56.095185 16282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9 [sys.catalog]: This master's current role is: LEADER
I20260620 18:43:56.095535 16288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:43:56.096166 16288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:43:56.098151 16288 catalog_manager.cc:1347] Generated new cluster ID: 653c6e3bf2d84408a63a6b64bc2a64f3
I20260620 18:43:56.098592 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.087648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.15.77.126:32991)
0620 18:43:56.087875 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.087885 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.087917 (+ 32us) server_negotiation.cc:407] Connection header received
0620 18:43:56.088386 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.088396 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.088461 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.088615 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:56.089224 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.090129 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.090901 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.091429 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.091569 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.093989 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:56.093994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:56.093994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:56.094017 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:56.096305 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.096772 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.096776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.096777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.096948 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.097257 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.097258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.097258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.097572 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:56.097712 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.098250 (+ 538us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.098393 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":124,"threads_started":1}
I20260620 18:43:56.099555 16288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:43:56.114429 16288 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:43:56.115020 16288 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:43:56.127534 16288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6628211d428b40b283ed47e19a0fa7d9: Generated new TSK 0
I20260620 18:43:56.127792 16288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:43:56.162251 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:32991
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:56.248399 16301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:56.248637 16301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:56.248664 16301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:56.249948 16301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:56.250007 16301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:56.250039 16301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:43:56.251725 16301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:32991
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:56.252009 16301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:56.252280 16301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:43:56.257550 16309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:56.257587 16307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:56.257614 16306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:56.258067 16301 server_base.cc:1034] running on GCE node
I20260620 18:43:56.258279 16301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:56.258584 16301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:56.259788 16301 hybrid_clock.cc:648] HybridClock initialized: now 1781981036259776 us; error 37 us; skew 500 ppm
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:56.262482 16301 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:43:56.262967 16301 webserver.cc:460] Webserver started at http://127.15.77.65:44527/ using document root <none> and password file <none>
I20260620 18:43:56.263165 16301 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:56.263208 16301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:56.263302 16301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:56.264422 16301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "b0cf7c5447db4aea9ce26ed2509a170d"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "a70568fc91f95f995bcf73ccc3c011a1"
server_key_iv: "fb63cc96236e961607f77d0ae6389461"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.264748 16301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "b0cf7c5447db4aea9ce26ed2509a170d"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "a70568fc91f95f995bcf73ccc3c011a1"
server_key_iv: "fb63cc96236e961607f77d0ae6389461"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.266100 16301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:43:56.266882 16316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:56.267128 16301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:43:56.267247 16301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "b0cf7c5447db4aea9ce26ed2509a170d"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "a70568fc91f95f995bcf73ccc3c011a1"
server_key_iv: "fb63cc96236e961607f77d0ae6389461"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.267372 16301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:56.274714 16301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:56.277281 16301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:56.277446 16301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:56.277688 16301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:43:56.278057 16301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:43:56.278095 16301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:56.278137 16301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:43:56.278165 16301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:56.284240 16301 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:35433
I20260620 18:43:56.284329 16429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:35433 every 8 connection(s)
I20260620 18:43:56.284705 16301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:43:56.287499 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 16301
I20260620 18:43:56.287606 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:43:56.287768 15669 external_mini_cluster.cc:1428] Setting key 8d2f42d6bbd375b371e559e6e9ea3b8b
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:43:56.296682 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.285535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:51165 (local address 127.15.77.126:32991)
0620 18:43:56.285638 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.285641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.285843 (+ 202us) server_negotiation.cc:407] Connection header received
0620 18:43:56.286867 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.286868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.286882 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.286936 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:56.287907 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.288688 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.290280 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.290627 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.290711 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.293358 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:56.293360 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:56.293361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:56.293371 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:56.295046 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.295577 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.295578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.295579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.295643 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.295908 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.295909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.295909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.296108 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:56.296186 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.296459 (+ 273us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.296558 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 18:43:56.296862 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.285516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:32991 (local address 127.15.77.65:51165)
0620 18:43:56.285702 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:56.285751 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:56.286732 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:56.287019 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:56.287023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:56.287441 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:56.287767 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.287774 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.288795 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.288796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.290186 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.290189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.290756 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.290757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.290879 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.291502 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:56.291513 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:56.293247 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:56.295163 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:56.295172 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:56.295175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:56.295484 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:56.295724 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:56.295725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:56.295725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:56.295826 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:56.296173 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:56.296179 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:56.296308 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 18:43:56.296393 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 18:43:56.296529 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260620 18:43:56.297407 16430 heartbeater.cc:347] Connected to a master server at 127.15.77.126:32991
I20260620 18:43:56.297499 16430 heartbeater.cc:464] Registering TS with master...
I20260620 18:43:56.297770 16430 heartbeater.cc:511] Master 127.15.77.126:32991 requested a full tablet report, sending...
I20260620 18:43:56.298319 16239 ts_manager.cc:194] Registered new tserver with Master: b0cf7c5447db4aea9ce26ed2509a170d (127.15.77.65:35433)
I20260620 18:43:56.298964 16239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:51165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:43:56.314684 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:32991
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:56.398738 16435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:56.398955 16435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:56.398984 16435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:56.400154 16435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:56.400195 16435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:56.400223 16435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:43:56.401651 16435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:32991
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:56.401846 16435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:56.402020 16435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:43:56.407263 16441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:56.407357 16440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:56.407279 16443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:56.407490 16435 server_base.cc:1034] running on GCE node
I20260620 18:43:56.407784 16435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:56.408039 16435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:56.409195 16435 hybrid_clock.cc:648] HybridClock initialized: now 1781981036409171 us; error 40 us; skew 500 ppm
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:56.411844 16435 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:43:56.412312 16435 webserver.cc:460] Webserver started at http://127.15.77.66:46697/ using document root <none> and password file <none>
I20260620 18:43:56.412518 16435 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:56.412561 16435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:56.412653 16435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:56.413664 16435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "02be5e99a31c48aabc70bc841c60b841"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "75b51217af2699f26ce94a3b18cd0221"
server_key_iv: "bb2ac3e3f52c377e4042c078a9c0c45b"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.413977 16435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "02be5e99a31c48aabc70bc841c60b841"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "75b51217af2699f26ce94a3b18cd0221"
server_key_iv: "bb2ac3e3f52c377e4042c078a9c0c45b"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.415282 16435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:43:56.416091 16450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:56.416267 16435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:43:56.416357 16435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "02be5e99a31c48aabc70bc841c60b841"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "75b51217af2699f26ce94a3b18cd0221"
server_key_iv: "bb2ac3e3f52c377e4042c078a9c0c45b"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.416415 16435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:56.439020 16435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:56.441290 16435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:56.441411 16435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:56.441604 16435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:43:56.441910 16435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:43:56.441939 16435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:56.441962 16435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:43:56.441975 16435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:56.447589 16435 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:44183
I20260620 18:43:56.447647 16563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:44183 every 8 connection(s)
I20260620 18:43:56.447973 16435 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:43:56.450170 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 16435
I20260620 18:43:56.450248 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:43:56.450428 15669 external_mini_cluster.cc:1428] Setting key 5f9f383d850cb3d846c3601132e7280b
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
I20260620 18:43:56.459067 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.448689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:37217 (local address 127.15.77.126:32991)
0620 18:43:56.448754 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.448756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.448942 (+ 186us) server_negotiation.cc:407] Connection header received
0620 18:43:56.450045 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.450046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.450060 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.450109 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:56.450935 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.451763 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.453104 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.453302 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.453355 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.455988 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:56.455990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:56.455990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:56.455999 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:56.457484 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.458021 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.458023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.458023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.458092 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.458375 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.458376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.458376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.458501 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:56.458578 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.458875 (+ 297us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.458962 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 18:43:56.459280 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.448670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:32991 (local address 127.15.77.66:37217)
0620 18:43:56.448821 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:56.448857 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:56.449870 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:56.450186 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:56.450189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:56.450556 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:56.450852 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.450859 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.451884 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.451885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.452993 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.452996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.453380 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.453381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.453545 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.454147 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:56.454158 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:56.455883 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:56.457623 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:56.457631 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:56.457633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:56.457931 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:56.458162 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:56.458163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:56.458163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:56.458298 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:56.458572 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:56.458578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:56.458727 (+ 149us) client_negotiation.cc:769] Sending connection context
0620 18:43:56.458843 (+ 116us) client_negotiation.cc:240] Negotiation successful
0620 18:43:56.459012 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260620 18:43:56.459787 16564 heartbeater.cc:347] Connected to a master server at 127.15.77.126:32991
I20260620 18:43:56.459892 16564 heartbeater.cc:464] Registering TS with master...
I20260620 18:43:56.460052 16564 heartbeater.cc:511] Master 127.15.77.126:32991 requested a full tablet report, sending...
I20260620 18:43:56.460491 16239 ts_manager.cc:194] Registered new tserver with Master: 02be5e99a31c48aabc70bc841c60b841 (127.15.77.66:44183)
I20260620 18:43:56.460850 16239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:37217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:43:56.476465 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:32991
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:56.559835 16569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:56.560062 16569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:56.560089 16569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:56.561252 16569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:56.561291 16569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:56.561318 16569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:43:56.562768 16569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:32991
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:56.562976 16569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:56.563166 16569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:43:56.568365 16569 server_base.cc:1034] running on GCE node
W20260620 18:43:56.568405 16575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:56.568382 16577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:56.568490 16574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:56.568820 16569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:56.569034 16569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:56.570161 16569 hybrid_clock.cc:648] HybridClock initialized: now 1781981036570143 us; error 30 us; skew 500 ppm
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:56.572934 16569 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:43:56.573335 16569 webserver.cc:460] Webserver started at http://127.15.77.67:35993/ using document root <none> and password file <none>
I20260620 18:43:56.573530 16569 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:56.573571 16569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:56.573657 16569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:56.574680 16569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "2de0c1ac4fd399c54364333530ae3197"
server_key_iv: "66c9602c83ec17d3859e1b356af01803"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.574988 16569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "2de0c1ac4fd399c54364333530ae3197"
server_key_iv: "66c9602c83ec17d3859e1b356af01803"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.576434 16569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:43:56.577209 16584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:56.577390 16569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:43:56.577482 16569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3"
format_stamp: "Formatted at 2026-06-20 18:43:56 on dist-test-slave-tvl9"
server_key: "2de0c1ac4fd399c54364333530ae3197"
server_key_iv: "66c9602c83ec17d3859e1b356af01803"
server_key_version: "encryptionkey@0"
I20260620 18:43:56.577539 16569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:56.612154 16569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:56.614590 16569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:56.614711 16569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:56.614913 16569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:43:56.615259 16569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:43:56.615295 16569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:56.615368 16569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:43:56.615401 16569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:56.621436 16569 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:41571
I20260620 18:43:56.621497 16697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:41571 every 8 connection(s)
I20260620 18:43:56.621840 16569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:56.631826 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 16569
I20260620 18:43:56.631870 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.622710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:56633 (local address 127.15.77.126:32991)
0620 18:43:56.622782 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.622785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.623025 (+ 240us) server_negotiation.cc:407] Connection header received
0620 18:43:56.624061 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.624063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.624080 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.624159 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:56.624844 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.625382 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.626385 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.626599 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.626658 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.628952 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:56.628954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:56.628954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:56.628960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:56.630423 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.630857 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.630858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.630858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.630909 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.631127 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.631128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.631128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.631232 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:56.631282 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.631676 (+ 394us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.631759 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 18:43:56.631923 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:43:56.632000 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.622695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:32991 (local address 127.15.77.67:56633)
0620 18:43:56.622895 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:56.622935 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:56.623940 (+ 1005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:56.624233 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:56.624237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:56.624505 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:56.624752 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.624756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.625468 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.625469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.626301 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.626303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.626676 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.626677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.626782 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.627389 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:56.627406 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:56.628863 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:56.630513 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:56.630520 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:56.630522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:56.630787 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:56.630968 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:56.630968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:56.630968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:56.631072 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:56.631288 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:56.631293 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:56.631507 (+ 214us) client_negotiation.cc:769] Sending connection context
0620 18:43:56.631627 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 18:43:56.631775 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260620 18:43:56.632086 15669 external_mini_cluster.cc:1428] Setting key 07caeb8665f9b3ef694e191f1a841bbd
I20260620 18:43:56.632501 16698 heartbeater.cc:347] Connected to a master server at 127.15.77.126:32991
I20260620 18:43:56.632624 16698 heartbeater.cc:464] Registering TS with master...
I20260620 18:43:56.632812 16698 heartbeater.cc:511] Master 127.15.77.126:32991 requested a full tablet report, sending...
I20260620 18:43:56.633193 16238 ts_manager.cc:194] Registered new tserver with Master: 8f1fa6db7f4f4695825a8bf7dd0531a3 (127.15.77.67:41571)
I20260620 18:43:56.633603 16238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:56633
I20260620 18:43:56.643953 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:43:56.651830 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.645615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.15.77.126:32991)
0620 18:43:56.645683 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.645687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.645801 (+ 114us) server_negotiation.cc:407] Connection header received
0620 18:43:56.645834 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.645835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.645846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.645912 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:56.646438 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.646925 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.647727 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.647912 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.647970 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.649101 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:56.649102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:56.649102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:56.649109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:56.650569 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.650869 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.650869 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.650870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.650919 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.651168 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.651169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.651169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.651396 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:56.651487 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.651698 (+ 211us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.651764 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 18:43:56.654367 16238 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:43:56.656256 16238 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:43:56.664013 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.658957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:35433 (local address 127.0.0.1:49536)
0620 18:43:56.659364 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:56.659405 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:56.659530 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:56.660516 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:56.660517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:56.660519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:56.660576 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.660578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.661920 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.661921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.662782 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.662785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.663691 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.663692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.663771 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.663784 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 18:43:56.663854 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 18:43:56.663915 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":52,"threads_started":1}
I20260620 18:43:56.664196 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.660009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49536 (local address 127.15.77.65:35433)
0620 18:43:56.660330 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.660334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.660345 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:43:56.660387 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.660389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.660406 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.660668 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:56.660754 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.661788 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.662868 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.663602 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.663680 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.663741 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.664060 (+ 319us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.664107 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":94,"threads_started":1}
I20260620 18:43:56.664014 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.659024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:44183 (local address 127.0.0.1:37814)
0620 18:43:56.659823 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:56.659836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:56.659902 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:56.660131 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:56.660135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:56.660137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:56.660237 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.660240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.661565 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.661566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.662951 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.662953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.663815 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.663816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.663874 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.663877 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:43:56.663920 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 18:43:56.663950 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":70,"thread_start_us":127,"threads_started":1}
I20260620 18:43:56.664518 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.659354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37814 (local address 127.15.77.66:44183)
0620 18:43:56.659559 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.659564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.659892 (+ 328us) server_negotiation.cc:407] Connection header received
0620 18:43:56.659973 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.659976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.659992 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.660076 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:56.660330 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.661452 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.663049 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.663637 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.663726 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.663783 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.664376 (+ 593us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.664418 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":113,"threads_started":1}
I20260620 18:43:56.664644 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.659636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55790 (local address 127.15.77.67:41571)
0620 18:43:56.659816 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.659821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.660332 (+ 511us) server_negotiation.cc:407] Connection header received
0620 18:43:56.660378 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.660380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.660398 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.660464 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:56.660678 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.661684 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.663047 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.663789 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.663872 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.663925 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.664516 (+ 591us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.664562 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260620 18:43:56.664989 16364 tablet_service.cc:1467] Processing CreateTablet for tablet 0aeb9bd0a35f41eb965096fc444c9637 (DEFAULT_TABLE table=test-table [id=f6a38a5f182d407ab599f3ee52e2a947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:43:56.665141 16498 tablet_service.cc:1467] Processing CreateTablet for tablet 0aeb9bd0a35f41eb965096fc444c9637 (DEFAULT_TABLE table=test-table [id=f6a38a5f182d407ab599f3ee52e2a947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:43:56.665408 16364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aeb9bd0a35f41eb965096fc444c9637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:56.665408 16498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aeb9bd0a35f41eb965096fc444c9637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:56.668891 16716 tablet_bootstrap.cc:492] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841: Bootstrap starting.
I20260620 18:43:56.668947 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.659161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:41571 (local address 127.0.0.1:55790)
0620 18:43:56.660211 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:56.660221 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:56.660274 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:56.660547 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:56.660548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:56.660550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:56.660604 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.660606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.661786 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.661787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.662950 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.662953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.663960 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.663962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.664022 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.664027 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:43:56.664076 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 18:43:56.668856 (+ 4780us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":50,"thread_start_us":85,"threads_started":1}
I20260620 18:43:56.669667 16632 tablet_service.cc:1467] Processing CreateTablet for tablet 0aeb9bd0a35f41eb965096fc444c9637 (DEFAULT_TABLE table=test-table [id=f6a38a5f182d407ab599f3ee52e2a947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:43:56.669988 16632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aeb9bd0a35f41eb965096fc444c9637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:56.670012 16716 tablet_bootstrap.cc:654] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:56.670495 16716 log.cc:824] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841: Log is configured to *not* fsync() on all Append() calls
I20260620 18:43:56.670883 16715 tablet_bootstrap.cc:492] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d: Bootstrap starting.
I20260620 18:43:56.671298 16716 tablet_bootstrap.cc:492] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841: No bootstrap required, opened a new log
I20260620 18:43:56.671404 16716 ts_tablet_manager.cc:1397] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 18:43:56.671582 16715 tablet_bootstrap.cc:654] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:56.672036 16715 log.cc:824] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:43:56.672791 16715 tablet_bootstrap.cc:492] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d: No bootstrap required, opened a new log
I20260620 18:43:56.672860 16715 ts_tablet_manager.cc:1397] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:43:56.673316 16719 tablet_bootstrap.cc:492] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3: Bootstrap starting.
I20260620 18:43:56.673576 16716 raft_consensus.cc:348] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be5e99a31c48aabc70bc841c60b841" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44183 } } peers { permanent_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35433 } } peers { permanent_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41571 } }
I20260620 18:43:56.673743 16716 raft_consensus.cc:374] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:56.673786 16716 raft_consensus.cc:729] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02be5e99a31c48aabc70bc841c60b841, State: Initialized, Role: FOLLOWER
I20260620 18:43:56.673893 16716 consensus_queue.cc:260] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be5e99a31c48aabc70bc841c60b841" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44183 } } peers { permanent_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35433 } } peers { permanent_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41571 } }
I20260620 18:43:56.673987 16719 tablet_bootstrap.cc:654] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:56.674189 16716 ts_tablet_manager.cc:1428] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 18:43:56.674201 16564 heartbeater.cc:503] Master 127.15.77.126:32991 was elected leader, sending a full tablet report...
I20260620 18:43:56.674530 16719 log.cc:824] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3: Log is configured to *not* fsync() on all Append() calls
I20260620 18:43:56.675000 16715 raft_consensus.cc:348] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be5e99a31c48aabc70bc841c60b841" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44183 } } peers { permanent_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35433 } } peers { permanent_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41571 } }
I20260620 18:43:56.675194 16715 raft_consensus.cc:374] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:56.675240 16715 raft_consensus.cc:729] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0cf7c5447db4aea9ce26ed2509a170d, State: Initialized, Role: FOLLOWER
I20260620 18:43:56.675278 16719 tablet_bootstrap.cc:492] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3: No bootstrap required, opened a new log
I20260620 18:43:56.675366 16719 ts_tablet_manager.cc:1397] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:43:56.675402 16715 consensus_queue.cc:260] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be5e99a31c48aabc70bc841c60b841" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44183 } } peers { permanent_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35433 } } peers { permanent_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41571 } }
I20260620 18:43:56.675704 16715 ts_tablet_manager.cc:1428] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:43:56.675745 16430 heartbeater.cc:503] Master 127.15.77.126:32991 was elected leader, sending a full tablet report...
I20260620 18:43:56.677314 16719 raft_consensus.cc:348] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be5e99a31c48aabc70bc841c60b841" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44183 } } peers { permanent_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35433 } } peers { permanent_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41571 } }
I20260620 18:43:56.677438 16719 raft_consensus.cc:374] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:56.677459 16719 raft_consensus.cc:729] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f1fa6db7f4f4695825a8bf7dd0531a3, State: Initialized, Role: FOLLOWER
I20260620 18:43:56.677532 16719 consensus_queue.cc:260] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be5e99a31c48aabc70bc841c60b841" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44183 } } peers { permanent_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35433 } } peers { permanent_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41571 } }
I20260620 18:43:56.677768 16719 ts_tablet_manager.cc:1428] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:43:56.677783 16698 heartbeater.cc:503] Master 127.15.77.126:32991 was elected leader, sending a full tablet report...
W20260620 18:43:56.698498 16565 tablet.cc:2369] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:43:56.776069 16723 raft_consensus.cc:482] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:43:56.776201 16723 raft_consensus.cc:504] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be5e99a31c48aabc70bc841c60b841" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44183 } } peers { permanent_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35433 } } peers { permanent_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41571 } }
I20260620 18:43:56.776656 16723 leader_election.cc:288] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02be5e99a31c48aabc70bc841c60b841 (127.15.77.66:44183), b0cf7c5447db4aea9ce26ed2509a170d (127.15.77.65:35433)
I20260620 18:43:56.780668 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.777049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:53107 (local address 127.15.77.65:35433)
0620 18:43:56.777094 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.777096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.777189 (+ 93us) server_negotiation.cc:407] Connection header received
0620 18:43:56.777302 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.777304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.777317 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.777374 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:56.777547 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.778213 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.779662 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.780276 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.780344 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.780352 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.780545 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.780586 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:43:56.780704 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.776927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:35433 (local address 127.15.77.67:53107)
0620 18:43:56.777128 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:56.777139 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:56.777222 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:56.777396 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:56.777396 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:56.777398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:56.777466 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.777467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.778288 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.778289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.779554 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.779557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.780395 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.780396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.780454 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.780458 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:43:56.780540 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 18:43:56.780586 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":119,"threads_started":1}
I20260620 18:43:56.780735 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.776916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:48625 (local address 127.15.77.66:44183)
0620 18:43:56.776983 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.776986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.776997 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:43:56.777096 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.777096 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.777110 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.777183 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:56.777352 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.778124 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.779663 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.780246 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.780335 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.780347 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.780601 (+ 254us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.780664 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 18:43:56.780987 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.776884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:44183 (local address 127.15.77.67:48625)
0620 18:43:56.776943 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:56.776958 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:56.777007 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:56.777186 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:56.777187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:56.777189 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:56.777265 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.777267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.778216 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.778216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.779554 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:56.779557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.780378 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.780379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:56.780459 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.780463 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:43:56.780590 (+ 127us) client_negotiation.cc:240] Negotiation successful
0620 18:43:56.780639 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":6}
I20260620 18:43:56.781194 16384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aeb9bd0a35f41eb965096fc444c9637" candidate_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" is_pre_election: true
I20260620 18:43:56.781383 16384 raft_consensus.cc:2454] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f1fa6db7f4f4695825a8bf7dd0531a3 in term 0.
I20260620 18:43:56.781370 16518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aeb9bd0a35f41eb965096fc444c9637" candidate_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02be5e99a31c48aabc70bc841c60b841" is_pre_election: true
I20260620 18:43:56.781605 16518 raft_consensus.cc:2454] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f1fa6db7f4f4695825a8bf7dd0531a3 in term 0.
I20260620 18:43:56.781689 16586 leader_election.cc:302] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f1fa6db7f4f4695825a8bf7dd0531a3, b0cf7c5447db4aea9ce26ed2509a170d; no voters:
I20260620 18:43:56.781827 16723 raft_consensus.cc:2781] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:43:56.781903 16723 raft_consensus.cc:482] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:43:56.781949 16723 raft_consensus.cc:3037] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:56.782675 16723 raft_consensus.cc:504] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be5e99a31c48aabc70bc841c60b841" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44183 } } peers { permanent_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35433 } } peers { permanent_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41571 } }
I20260620 18:43:56.782861 16723 leader_election.cc:288] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [CANDIDATE]: Term 1 election: Requested vote from peers 02be5e99a31c48aabc70bc841c60b841 (127.15.77.66:44183), b0cf7c5447db4aea9ce26ed2509a170d (127.15.77.65:35433)
I20260620 18:43:56.783095 16518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aeb9bd0a35f41eb965096fc444c9637" candidate_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02be5e99a31c48aabc70bc841c60b841"
I20260620 18:43:56.783075 16384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aeb9bd0a35f41eb965096fc444c9637" candidate_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0cf7c5447db4aea9ce26ed2509a170d"
I20260620 18:43:56.783167 16518 raft_consensus.cc:3037] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:56.783169 16384 raft_consensus.cc:3037] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:56.784116 16518 raft_consensus.cc:2454] T 0aeb9bd0a35f41eb965096fc444c9637 P 02be5e99a31c48aabc70bc841c60b841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f1fa6db7f4f4695825a8bf7dd0531a3 in term 1.
I20260620 18:43:56.784116 16384 raft_consensus.cc:2454] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f1fa6db7f4f4695825a8bf7dd0531a3 in term 1.
I20260620 18:43:56.784296 16586 leader_election.cc:302] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f1fa6db7f4f4695825a8bf7dd0531a3, b0cf7c5447db4aea9ce26ed2509a170d; no voters:
I20260620 18:43:56.784422 16723 raft_consensus.cc:2781] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:43:56.784603 16723 raft_consensus.cc:686] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f1fa6db7f4f4695825a8bf7dd0531a3, State: Running, Role: LEADER
I20260620 18:43:56.784713 16723 consensus_queue.cc:237] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be5e99a31c48aabc70bc841c60b841" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44183 } } peers { permanent_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35433 } } peers { permanent_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41571 } }
W20260620 18:43:56.785420 16431 tablet.cc:2369] T 0aeb9bd0a35f41eb965096fc444c9637 P b0cf7c5447db4aea9ce26ed2509a170d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:43:56.785604 16238 catalog_manager.cc:5495] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f1fa6db7f4f4695825a8bf7dd0531a3 (127.15.77.67). New cstate: current_term: 1 leader_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be5e99a31c48aabc70bc841c60b841" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0cf7c5447db4aea9ce26ed2509a170d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f1fa6db7f4f4695825a8bf7dd0531a3" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41571 } health_report { overall_health: HEALTHY } } }
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:43:56.815164 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.807566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.15.77.65:35433)
0620 18:43:56.807626 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.807628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.807770 (+ 142us) server_negotiation.cc:407] Connection header received
0620 18:43:56.807799 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.807800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.807811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.807856 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:56.808389 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.808873 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.809611 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.809801 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.809855 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.812284 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:56.812298 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:56.812298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:56.812305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:56.813880 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.814230 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.814233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.814233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.814296 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.814541 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.814542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.814542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.814758 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:56.814876 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.815017 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.815093 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:43:56.838953 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.831784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.15.77.65:35433)
0620 18:43:56.831872 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.831875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.831919 (+ 44us) server_negotiation.cc:407] Connection header received
0620 18:43:56.831979 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.831980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.831989 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.832043 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:56.832640 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.833114 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.833853 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.834038 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.834097 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.836371 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:56.836372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:56.836373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:56.836379 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:56.837705 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.838077 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.838078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.838079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.838136 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.838318 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.838318 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.838319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.838420 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:56.838492 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.838705 (+ 213us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.838813 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:43:56.862998 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:56.855388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.15.77.65:35433)
0620 18:43:56.855475 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:56.855477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:56.855582 (+ 105us) server_negotiation.cc:407] Connection header received
0620 18:43:56.855616 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:56.855617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:56.855629 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:56.855688 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:56.856289 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.856770 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.857561 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:56.857779 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:56.857886 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:56.860309 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:56.860311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:56.860311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:56.860318 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:56.861721 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.862080 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.862081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.862081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.862132 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:56.862335 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:56.862336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:56.862337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:56.862449 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:56.862579 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:56.862753 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 18:43:56.862829 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260620 18:43:56.863626 16344 tablet_service.cc:2908] Rejecting scan request for tablet 0aeb9bd0a35f41eb965096fc444c9637: Uninitialized: clean time has not yet been initialized
I20260620 18:43:56.864495 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 16301
I20260620 18:43:56.869956 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 16435
W20260620 18:43:56.872426 16699 tablet.cc:2369] T 0aeb9bd0a35f41eb965096fc444c9637 P 8f1fa6db7f4f4695825a8bf7dd0531a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:43:56.874773 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 16569
I20260620 18:43:56.879204 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 16207
2026-06-20T18:43:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2091 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16739](info): set up 2 sockets
Jun 20 18:43:56 dist-test-slave-tvl9 krb5kdc[16739](info): commencing operation
krb5kdc: starting...
W20260620 18:43:57.926955 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:43:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:43:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:43:58.051158 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:41739
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:41739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:58.136145 16752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:58.136359 16752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:58.136384 16752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:58.137569 16752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:43:58.137612 16752 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 18:43:58.137625 16752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:58.137638 16752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:58.137651 16752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:43:58.137663 16752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:43:58.139168 16752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:41739
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:41739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:58.139479 16752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:58.139726 16752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:43:58.144659 16757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:58.144793 16752 server_base.cc:1034] running on GCE node
W20260620 18:43:58.144668 16758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:58.144733 16760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:58.145200 16752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:58.145522 16752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:58.146716 16752 hybrid_clock.cc:648] HybridClock initialized: now 1781981038146705 us; error 30 us; skew 500 ppm
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:58.149500 16752 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:43:58.150072 16752 webserver.cc:460] Webserver started at http://127.15.77.126:45939/ using document root <none> and password file <none>
I20260620 18:43:58.150326 16752 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:58.150403 16752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:58.150575 16752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:58.151850 16752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "f00d5c7713d44075b03dd5e338b42e15"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "d6d51e5c0f24380badc01b208db79460"
server_key_iv: "34147ca339d00c8fadf0c07084650785"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.152247 16752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "f00d5c7713d44075b03dd5e338b42e15"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "d6d51e5c0f24380badc01b208db79460"
server_key_iv: "34147ca339d00c8fadf0c07084650785"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.153699 16752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:43:58.154472 16767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.154680 16752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:43:58.154798 16752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "f00d5c7713d44075b03dd5e338b42e15"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "d6d51e5c0f24380badc01b208db79460"
server_key_iv: "34147ca339d00c8fadf0c07084650785"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.154879 16752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:58.164630 16752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:58.166882 16752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:58.167054 16752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:58.171986 16752 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:41739
I20260620 18:43:58.172055 16829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:41739 every 8 connection(s)
I20260620 18:43:58.172366 16752 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:43:58.173044 16830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:58.175253 16830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15: Bootstrap starting.
I20260620 18:43:58.175889 16830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:58.176146 16830 log.cc:824] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15: Log is configured to *not* fsync() on all Append() calls
I20260620 18:43:58.176570 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 16752
I20260620 18:43:58.176666 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:43:58.176833 15669 external_mini_cluster.cc:1428] Setting key fcff3476250e122187ea310aa79dbe4a
I20260620 18:43:58.176900 16830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15: No bootstrap required, opened a new log
I20260620 18:43:58.179831 16830 raft_consensus.cc:348] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00d5c7713d44075b03dd5e338b42e15" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41739 } }
I20260620 18:43:58.179973 16830 raft_consensus.cc:374] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:58.179997 16830 raft_consensus.cc:729] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00d5c7713d44075b03dd5e338b42e15, State: Initialized, Role: FOLLOWER
I20260620 18:43:58.180133 16830 consensus_queue.cc:260] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00d5c7713d44075b03dd5e338b42e15" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41739 } }
I20260620 18:43:58.180207 16830 raft_consensus.cc:388] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:43:58.180251 16830 raft_consensus.cc:482] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:43:58.180299 16830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:58.181031 16830 raft_consensus.cc:504] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00d5c7713d44075b03dd5e338b42e15" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41739 } }
I20260620 18:43:58.181143 16830 leader_election.cc:302] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f00d5c7713d44075b03dd5e338b42e15; no voters:
I20260620 18:43:58.181311 16830 leader_election.cc:288] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:43:58.181380 16835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:43:58.181571 16835 raft_consensus.cc:686] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [term 1 LEADER]: Becoming Leader. State: Replica: f00d5c7713d44075b03dd5e338b42e15, State: Running, Role: LEADER
I20260620 18:43:58.181597 16830 sys_catalog.cc:564] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:43:58.181655 16835 consensus_queue.cc:237] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00d5c7713d44075b03dd5e338b42e15" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41739 } }
I20260620 18:43:58.181984 16835 sys_catalog.cc:455] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f00d5c7713d44075b03dd5e338b42e15. Latest consensus state: current_term: 1 leader_uuid: "f00d5c7713d44075b03dd5e338b42e15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00d5c7713d44075b03dd5e338b42e15" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41739 } } }
I20260620 18:43:58.181995 16837 sys_catalog.cc:455] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f00d5c7713d44075b03dd5e338b42e15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f00d5c7713d44075b03dd5e338b42e15" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41739 } } }
I20260620 18:43:58.182060 16835 sys_catalog.cc:458] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [sys.catalog]: This master's current role is: LEADER
I20260620 18:43:58.182093 16837 sys_catalog.cc:458] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15 [sys.catalog]: This master's current role is: LEADER
I20260620 18:43:58.182452 16841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:43:58.183017 16841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:58.186328 16841 catalog_manager.cc:1347] Generated new cluster ID: cea849acfdb84bed917da48052234dd7
I20260620 18:43:58.186385 16841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:43:58.189059 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.177383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37710 (local address 127.15.77.126:41739)
0620 18:43:58.177554 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.177564 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.177692 (+ 128us) server_negotiation.cc:407] Connection header received
0620 18:43:58.178224 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.178231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.178289 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.178405 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:58.179221 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.180412 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.181441 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.182069 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.182180 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.185309 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:58.185313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:58.185314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:58.185326 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:58.187430 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.187850 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.187854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.187854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.187948 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.188172 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.188173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.188174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.188427 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:58.188611 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.188718 (+ 107us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.188863 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260620 18:43:58.197853 16841 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:43:58.198412 16841 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:43:58.205879 16841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15: Generated new TSK 0
I20260620 18:43:58.206090 16841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:43:58.232496 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41739
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:58.317934 16856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:58.318153 16856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:58.318181 16856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:58.319443 16856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:58.319514 16856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:58.319535 16856 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 18:43:58.319556 16856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:43:58.321126 16856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.77.126:41739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:58.321431 16856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:58.321704 16856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:43:58.327654 16856 server_base.cc:1034] running on GCE node
W20260620 18:43:58.327790 16864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:58.327816 16861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:58.327844 16862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:58.328227 16856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:58.328514 16856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:58.329684 16856 hybrid_clock.cc:648] HybridClock initialized: now 1781981038329651 us; error 48 us; skew 500 ppm
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:58.332660 16856 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:43:58.333170 16856 webserver.cc:460] Webserver started at http://127.15.77.65:43853/ using document root <none> and password file <none>
I20260620 18:43:58.333441 16856 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:58.333489 16856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:58.333635 16856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:58.334759 16856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "acc3b0dff3e342538fe6bb04e8c79d0e"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "7574b4ff5c9241f425f40f7220620cf6"
server_key_iv: "70f1915d8cbbffe5a6329fb55a316c75"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.335136 16856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "acc3b0dff3e342538fe6bb04e8c79d0e"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "7574b4ff5c9241f425f40f7220620cf6"
server_key_iv: "70f1915d8cbbffe5a6329fb55a316c75"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.336697 16856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:43:58.337639 16871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.337867 16856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:43:58.337954 16856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "acc3b0dff3e342538fe6bb04e8c79d0e"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "7574b4ff5c9241f425f40f7220620cf6"
server_key_iv: "70f1915d8cbbffe5a6329fb55a316c75"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.338060 16856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:58.349308 16856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:58.351776 16856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:58.351938 16856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:58.352478 16856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:43:58.352520 16856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.352542 16856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:43:58.352556 16856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:58.360268 16856 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:42065
I20260620 18:43:58.360354 16986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:42065 every 8 connection(s)
I20260620 18:43:58.360707 16856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:43:58.363096 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.353072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:36813 (local address 127.15.77.126:41739)
0620 18:43:58.353128 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.353130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.353392 (+ 262us) server_negotiation.cc:407] Connection header received
0620 18:43:58.353872 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.353873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.353887 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.353986 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:58.354900 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.355800 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.356919 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.357301 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.357382 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.360039 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:58.360040 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:58.360041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:58.360055 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:58.361614 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.362077 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.362078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.362078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.362142 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.362405 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.362406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.362407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.362533 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:58.362613 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.362901 (+ 288us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.362995 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 18:43:58.363266 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.353078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41739 (local address 127.15.77.65:36813)
0620 18:43:58.353250 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.353287 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.353769 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.354051 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.354056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.354461 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:58.354780 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.354788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.355929 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.355930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.356799 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.356801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.357427 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.357428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.357598 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.358249 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:58.358272 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:58.359923 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:58.361715 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.361723 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.361725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.361977 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.362210 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.362211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.362212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.362329 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.362603 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:58.362610 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:58.362757 (+ 147us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.362865 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.363029 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260620 18:43:58.363894 16987 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41739
I20260620 18:43:58.364032 16987 heartbeater.cc:464] Registering TS with master...
I20260620 18:43:58.364223 16987 heartbeater.cc:511] Master 127.15.77.126:41739 requested a full tablet report, sending...
I20260620 18:43:58.364823 16784 ts_manager.cc:194] Registered new tserver with Master: acc3b0dff3e342538fe6bb04e8c79d0e (127.15.77.65:42065)
I20260620 18:43:58.365778 16784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:36813
I20260620 18:43:58.368095 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 16856
I20260620 18:43:58.368208 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:43:58.368367 15669 external_mini_cluster.cc:1428] Setting key 5f5e9ed576b86bde0fde25580a4826dc
I20260620 18:43:58.371140 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.365108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:58061 (local address 127.15.77.126:41739)
0620 18:43:58.365149 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.365151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.365218 (+ 67us) server_negotiation.cc:407] Connection header received
0620 18:43:58.365271 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.365272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.365281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.365348 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:58.365638 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.366301 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.367064 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.367269 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.367424 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.368435 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:58.368436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:58.368436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:58.368443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:58.370051 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.370384 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.370385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.370385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.370451 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.370704 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.370705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.370705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.370805 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:58.370857 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.371023 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.371078 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:43:58.371268 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.365040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41739 (local address 127.15.77.65:58061)
0620 18:43:58.365161 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.365171 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.365229 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.365351 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.365351 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.365506 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:58.365568 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.365569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.366407 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.366408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.366958 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.366960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.367432 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.367433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.367500 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.367989 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:58.367994 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:58.368349 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:58.370143 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.370144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.370144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.370312 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.370548 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.370549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.370549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.370632 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.370871 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:58.370873 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:58.370926 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.371004 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.371141 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:43:58.394953 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41739
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:58.477021 16996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:58.477250 16996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:58.477277 16996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:58.478384 16996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:58.478421 16996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:58.478441 16996 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 18:43:58.478458 16996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:43:58.480319 16996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.77.126:41739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:58.480543 16996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:58.480720 16996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:43:58.486006 17002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:58.486063 17004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:58.486240 16996 server_base.cc:1034] running on GCE node
W20260620 18:43:58.486009 17001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:58.486634 16996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:58.486850 16996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:58.487991 16996 hybrid_clock.cc:648] HybridClock initialized: now 1781981038487971 us; error 66 us; skew 500 ppm
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:58.490496 16996 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:43:58.490909 16996 webserver.cc:460] Webserver started at http://127.15.77.66:44309/ using document root <none> and password file <none>
I20260620 18:43:58.491120 16996 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:58.491163 16996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:58.491256 16996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:58.492357 16996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "29d4cc548b844b549739cf6394ce2f8b"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "9f673c58a60814d1a926f91f337c5b95"
server_key_iv: "b862071de06d302428b43a5f1088a8ef"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.492666 16996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "29d4cc548b844b549739cf6394ce2f8b"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "9f673c58a60814d1a926f91f337c5b95"
server_key_iv: "b862071de06d302428b43a5f1088a8ef"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.494020 16996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:43:58.494732 17011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.494903 16996 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:43:58.495014 16996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "29d4cc548b844b549739cf6394ce2f8b"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "9f673c58a60814d1a926f91f337c5b95"
server_key_iv: "b862071de06d302428b43a5f1088a8ef"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.495100 16996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:58.504107 16996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:58.506125 16996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:58.506313 16996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:58.506826 16996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:43:58.506894 16996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.506942 16996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:43:58.506963 16996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:58.514276 16996 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:36523
I20260620 18:43:58.514358 17126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:36523 every 8 connection(s)
I20260620 18:43:58.514706 16996 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:43:58.517343 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.507375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:60297 (local address 127.15.77.126:41739)
0620 18:43:58.507450 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.507454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.507617 (+ 163us) server_negotiation.cc:407] Connection header received
0620 18:43:58.508110 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.508112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.508126 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.508209 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:58.508937 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.509794 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.510861 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.511220 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.511302 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.514220 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:58.514222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:58.514223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:58.514230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:58.515897 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.516345 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.516347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.516347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.516411 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.516671 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.516672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.516673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.516804 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:58.516881 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.517162 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.517245 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 18:43:58.517544 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.507307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41739 (local address 127.15.77.66:60297)
0620 18:43:58.507483 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.507524 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.508008 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.508314 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.508319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.508647 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:58.508854 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.508859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.509923 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.509924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.510776 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.510778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.511351 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.511352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.511527 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.512086 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:58.512096 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:58.514111 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:58.516007 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.516014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.516017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.516251 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.516479 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.516480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.516481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.516604 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.516874 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:58.516880 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:58.517025 (+ 145us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.517142 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.517308 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260620 18:43:58.518078 17127 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41739
I20260620 18:43:58.518167 17127 heartbeater.cc:464] Registering TS with master...
I20260620 18:43:58.518360 17127 heartbeater.cc:511] Master 127.15.77.126:41739 requested a full tablet report, sending...
I20260620 18:43:58.518808 16783 ts_manager.cc:194] Registered new tserver with Master: 29d4cc548b844b549739cf6394ce2f8b (127.15.77.66:36523)
I20260620 18:43:58.519290 16783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:60297
I20260620 18:43:58.520068 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 16996
I20260620 18:43:58.520141 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:43:58.520269 15669 external_mini_cluster.cc:1428] Setting key b54d16728c223efb830cd335195671bf
I20260620 18:43:58.525701 17133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.519136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41739 (local address 127.15.77.66:59929)
0620 18:43:58.519281 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.519293 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.519380 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.519551 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.519551 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.519758 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:58.519828 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.519830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.520633 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.520634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.521331 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.521333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.521792 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.521793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.521849 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.522327 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:58.522331 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:58.522654 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:58.524627 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.524629 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.524629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.524798 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.525010 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.525010 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.525011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.525079 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.525446 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:58.525446 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:58.525487 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.525543 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.525625 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260620 18:43:58.525789 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.519229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:59929 (local address 127.15.77.126:41739)
0620 18:43:58.519270 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.519272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.519361 (+ 89us) server_negotiation.cc:407] Connection header received
0620 18:43:58.519481 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.519482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.519494 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.519545 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:58.519904 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.520553 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.521409 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.521713 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.521791 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.522742 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:58.522743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:58.522743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:58.522751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:58.524539 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.524876 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.524878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.524878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.524943 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.525147 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.525148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.525149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.525372 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:58.525452 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.525596 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.525688 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:43:58.547489 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41739
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:43:58.632054 17136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:43:58.632283 17136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:43:58.632309 17136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:43:58.633598 17136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:43:58.633653 17136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:43:58.633675 17136 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 18:43:58.633693 17136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:43:58.635253 17136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.77.126:41739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:43:58.635538 17136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:43:58.635746 17136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:43:58.640964 17141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:58.640976 17144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:43:58.640990 17142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:43:58.641167 17136 server_base.cc:1034] running on GCE node
I20260620 18:43:58.641377 17136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:43:58.641642 17136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:43:58.642784 17136 hybrid_clock.cc:648] HybridClock initialized: now 1781981038642758 us; error 43 us; skew 500 ppm
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:43:58.645326 17136 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:43:58.645816 17136 webserver.cc:460] Webserver started at http://127.15.77.67:37263/ using document root <none> and password file <none>
I20260620 18:43:58.646065 17136 fs_manager.cc:359] Metadata directory not provided
I20260620 18:43:58.646111 17136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:43:58.646203 17136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:43:58.647218 17136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "8e33100310e04de685a859c19c07db02"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "d745790c64e63d9b9fe854060cd654db"
server_key_iv: "c087faee50ca6f17495a791476078070"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.647621 17136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "8e33100310e04de685a859c19c07db02"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "d745790c64e63d9b9fe854060cd654db"
server_key_iv: "c087faee50ca6f17495a791476078070"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.648960 17136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:43:58.649736 17151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.649953 17136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:43:58.650048 17136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "8e33100310e04de685a859c19c07db02"
format_stamp: "Formatted at 2026-06-20 18:43:58 on dist-test-slave-tvl9"
server_key: "d745790c64e63d9b9fe854060cd654db"
server_key_iv: "c087faee50ca6f17495a791476078070"
server_key_version: "encryptionkey@0"
I20260620 18:43:58.650105 17136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:43:58.669764 17136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:43:58.671924 17136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:43:58.672039 17136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:43:58.672515 17136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:43:58.672600 17136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.672634 17136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:43:58.672660 17136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.679852 17136 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:41011
I20260620 18:43:58.679911 17266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:41011 every 8 connection(s)
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:58.680253 17136 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:43:58.682842 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 17136
I20260620 18:43:58.682942 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:43:58.683185 15669 external_mini_cluster.cc:1428] Setting key fd6f53264ecc17b1b5c27e2c26fc7ef1
I20260620 18:43:58.684253 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.673057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:40699 (local address 127.15.77.126:41739)
0620 18:43:58.673131 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.673134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.673330 (+ 196us) server_negotiation.cc:407] Connection header received
0620 18:43:58.673832 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.673833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.673846 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.673931 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:58.674826 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.675747 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.677512 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.677837 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.677903 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.680791 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:58.680792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:58.680793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:58.680800 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:58.682385 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.682988 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.682990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.682991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.683058 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.683462 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.683463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.683464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.683622 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:58.683700 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.684103 (+ 403us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.684189 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 18:43:58.684507 17160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.673020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41739 (local address 127.15.77.67:40699)
0620 18:43:58.673198 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.673239 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.673736 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.674028 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.674033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.674415 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:58.674714 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.674722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.675875 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.675876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.677411 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.677414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.677935 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.677937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.678103 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.678731 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:58.678744 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:58.680706 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:58.682493 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.682500 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.682503 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.682878 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.683119 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.683120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.683120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.683258 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.683703 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:58.683710 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:58.683908 (+ 198us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.684051 (+ 143us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.684244 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260620 18:43:58.685165 17267 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41739
I20260620 18:43:58.685254 17267 heartbeater.cc:464] Registering TS with master...
I20260620 18:43:58.685447 17267 heartbeater.cc:511] Master 127.15.77.126:41739 requested a full tablet report, sending...
I20260620 18:43:58.685833 16783 ts_manager.cc:194] Registered new tserver with Master: 8e33100310e04de685a859c19c07db02 (127.15.77.67:41011)
I20260620 18:43:58.686349 16783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:40699
I20260620 18:43:58.692283 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.686337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:35619 (local address 127.15.77.126:41739)
0620 18:43:58.686382 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.686384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.686466 (+ 82us) server_negotiation.cc:407] Connection header received
0620 18:43:58.686725 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.686726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.686735 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.686792 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:58.687210 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.687904 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.688650 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.688832 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.688887 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.689856 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:58.689858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:58.689858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:58.689864 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:58.691382 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.691709 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.691710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.691710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.691762 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.691890 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.691890 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.691890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.691983 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:58.692044 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.692160 (+ 116us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.692218 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:43:58.692288 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.686253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41739 (local address 127.15.77.67:35619)
0620 18:43:58.686395 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.686408 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.686491 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.686816 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.686817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.687020 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:58.687105 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.687107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.688025 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.688026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.688570 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.688572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.688896 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.688897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.689004 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.689457 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:58.689462 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:58.689779 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:58.691440 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.691440 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.691441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.691634 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.691811 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.691811 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.691812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.691856 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.692052 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:58.692053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:58.692091 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.692142 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.692221 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260620 18:43:58.695382 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:58.719504 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.712255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.15.77.126:41739)
0620 18:43:58.712325 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.712327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.712336 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:43:58.712362 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.712363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.712372 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.712424 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:58.713034 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.713567 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.714332 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.714560 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.714619 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.717025 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:58.717027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:58.717027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:58.717035 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:58.718427 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.718772 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.718773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.718773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.718824 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.718988 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.718989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.718989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.719097 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:58.719169 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.719351 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.719423 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 18:43:58.721246 16783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:43:58.722900 16783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:43:58.729650 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.725543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59586 (local address 127.15.77.66:36523)
0620 18:43:58.725749 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.725755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.725911 (+ 156us) server_negotiation.cc:407] Connection header received
0620 18:43:58.726015 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.726017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.726034 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.726145 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:58.726543 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.727234 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.728672 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.729182 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.729281 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.729345 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.729491 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.729530 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":84,"threads_started":1}
I20260620 18:43:58.729641 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.725441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:36523 (local address 127.0.0.1:59586)
0620 18:43:58.725802 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.725848 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.725926 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.726396 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.726397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.726398 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:58.726459 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.726461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.727357 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.727358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.728601 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.728604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.729282 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.729283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.729362 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.729376 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.729472 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.729530 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":54,"thread_start_us":63,"threads_started":1}
I20260620 18:43:58.729830 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.725919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.15.77.65:42065)
0620 18:43:58.726118 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.726123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.726136 (+ 13us) server_negotiation.cc:407] Connection header received
0620 18:43:58.726176 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.726178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.726192 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.726269 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:58.726523 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.727230 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.728506 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.729269 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.729340 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.729389 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.729685 (+ 296us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.729730 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260620 18:43:58.729871 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.725659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:42065 (local address 127.0.0.1:44762)
0620 18:43:58.726033 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.726042 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.726112 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.726342 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.726345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.726347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:58.726434 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.726436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.727362 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.727363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.728417 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.728419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.729421 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.729422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.729491 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.729495 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.729539 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.729581 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260620 18:43:58.730803 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.725441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:41011 (local address 127.0.0.1:57030)
0620 18:43:58.725802 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.725846 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.725925 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.726375 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.726376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.726378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:58.726444 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.726446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.728572 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.728572 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.729618 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.729620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.730497 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.730498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.730639 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.730643 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.730693 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.730738 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":21,"thread_start_us":74,"threads_started":1}
I20260620 18:43:58.730717 16921 tablet_service.cc:1467] Processing CreateTablet for tablet f8156699b4344c3eb1907d143ac76c53 (DEFAULT_TABLE table=test-table [id=3dd5aaec308447cd9b64fae4ebc8bffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:43:58.730940 17061 tablet_service.cc:1467] Processing CreateTablet for tablet f8156699b4344c3eb1907d143ac76c53 (DEFAULT_TABLE table=test-table [id=3dd5aaec308447cd9b64fae4ebc8bffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:43:58.731127 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.725661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57030 (local address 127.15.77.67:41011)
0620 18:43:58.725849 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.725853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.725937 (+ 84us) server_negotiation.cc:407] Connection header received
0620 18:43:58.726243 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.726246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.726262 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.726334 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:58.727368 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.728493 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.729707 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.730342 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.730412 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.730462 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.730977 (+ 515us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.731021 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
I20260620 18:43:58.731137 16921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8156699b4344c3eb1907d143ac76c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:58.731307 17061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8156699b4344c3eb1907d143ac76c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:58.731716 17201 tablet_service.cc:1467] Processing CreateTablet for tablet f8156699b4344c3eb1907d143ac76c53 (DEFAULT_TABLE table=test-table [id=3dd5aaec308447cd9b64fae4ebc8bffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:43:58.732012 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8156699b4344c3eb1907d143ac76c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:58.734338 17290 tablet_bootstrap.cc:492] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: Bootstrap starting.
I20260620 18:43:58.735038 17290 tablet_bootstrap.cc:654] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:58.735117 17291 tablet_bootstrap.cc:492] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: Bootstrap starting.
I20260620 18:43:58.735184 17292 tablet_bootstrap.cc:492] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: Bootstrap starting.
I20260620 18:43:58.735615 17290 log.cc:824] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: Log is configured to *not* fsync() on all Append() calls
I20260620 18:43:58.735803 17291 tablet_bootstrap.cc:654] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:58.736080 17291 log.cc:824] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: Log is configured to *not* fsync() on all Append() calls
I20260620 18:43:58.736214 17292 tablet_bootstrap.cc:654] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:58.736522 17292 log.cc:824] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: Log is configured to *not* fsync() on all Append() calls
I20260620 18:43:58.736660 17290 tablet_bootstrap.cc:492] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: No bootstrap required, opened a new log
I20260620 18:43:58.736729 17290 ts_tablet_manager.cc:1397] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 18:43:58.737258 17292 tablet_bootstrap.cc:492] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: No bootstrap required, opened a new log
I20260620 18:43:58.737336 17292 ts_tablet_manager.cc:1397] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:43:58.738258 17290 raft_consensus.cc:348] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } }
I20260620 18:43:58.738389 17290 raft_consensus.cc:374] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:58.738411 17290 raft_consensus.cc:729] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29d4cc548b844b549739cf6394ce2f8b, State: Initialized, Role: FOLLOWER
I20260620 18:43:58.738601 17290 consensus_queue.cc:260] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } }
I20260620 18:43:58.738934 17296 raft_consensus.cc:482] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:43:58.738932 17290 ts_tablet_manager.cc:1428] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:43:58.739037 17296 raft_consensus.cc:504] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } }
I20260620 18:43:58.738932 17292 raft_consensus.cc:348] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } }
I20260620 18:43:58.739097 17292 raft_consensus.cc:374] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:58.739110 17127 heartbeater.cc:503] Master 127.15.77.126:41739 was elected leader, sending a full tablet report...
I20260620 18:43:58.739131 17292 raft_consensus.cc:729] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e33100310e04de685a859c19c07db02, State: Initialized, Role: FOLLOWER
I20260620 18:43:58.739259 17292 consensus_queue.cc:260] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } }
I20260620 18:43:58.739554 17292 ts_tablet_manager.cc:1428] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:43:58.739678 17267 heartbeater.cc:503] Master 127.15.77.126:41739 was elected leader, sending a full tablet report...
I20260620 18:43:58.739688 17291 tablet_bootstrap.cc:492] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: No bootstrap required, opened a new log
I20260620 18:43:58.739753 17291 ts_tablet_manager.cc:1397] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260620 18:43:58.740595 17296 leader_election.cc:288] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e33100310e04de685a859c19c07db02 (127.15.77.67:41011), acc3b0dff3e342538fe6bb04e8c79d0e (127.15.77.65:42065)
I20260620 18:43:58.741936 17291 raft_consensus.cc:348] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } }
I20260620 18:43:58.742092 17291 raft_consensus.cc:374] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:58.742134 17291 raft_consensus.cc:729] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc3b0dff3e342538fe6bb04e8c79d0e, State: Initialized, Role: FOLLOWER
I20260620 18:43:58.742249 17291 consensus_queue.cc:260] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } }
I20260620 18:43:58.742529 17291 ts_tablet_manager.cc:1428] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:43:58.742532 16987 heartbeater.cc:503] Master 127.15.77.126:41739 was elected leader, sending a full tablet report...
I20260620 18:43:58.743552 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.740176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:38685 (local address 127.15.77.67:41011)
0620 18:43:58.740216 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.740218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.740227 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:43:58.740249 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.740250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.740261 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.740318 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:58.740519 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.741283 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.742633 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.743162 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.743227 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.743238 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.743450 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.743489 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 18:43:58.743564 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.739520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:41011 (local address 127.15.77.66:38685)
0620 18:43:58.739696 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.739708 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.739755 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.740369 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.740370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.740372 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:58.740437 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.740439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.741369 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.741369 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.742541 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.742543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.743224 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.743225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.743373 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.743378 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.743443 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.743492 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260620 18:43:58.744136 17221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8156699b4344c3eb1907d143ac76c53" candidate_uuid: "29d4cc548b844b549739cf6394ce2f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e33100310e04de685a859c19c07db02" is_pre_election: true
I20260620 18:43:58.744264 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.740524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:51077 (local address 127.15.77.65:42065)
0620 18:43:58.740576 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.740578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.740772 (+ 194us) server_negotiation.cc:407] Connection header received
0620 18:43:58.740893 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.740894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.740903 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.740930 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:58.741144 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.741856 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.743245 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.743780 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.743820 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.743826 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.744157 (+ 331us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.744191 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:43:58.744270 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.740349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:42065 (local address 127.15.77.66:51077)
0620 18:43:58.740712 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.740722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.740807 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.741014 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.741014 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.741016 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:58.741076 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.741077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.741971 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.741972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.743150 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.743152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.743983 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.743984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.744087 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.744091 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.744152 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.744191 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":70,"threads_started":1}
I20260620 18:43:58.744272 17221 raft_consensus.cc:2454] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29d4cc548b844b549739cf6394ce2f8b in term 0.
I20260620 18:43:58.744495 17015 leader_election.cc:302] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29d4cc548b844b549739cf6394ce2f8b, 8e33100310e04de685a859c19c07db02; no voters:
I20260620 18:43:58.744643 17297 raft_consensus.cc:2781] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:43:58.744634 16941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8156699b4344c3eb1907d143ac76c53" candidate_uuid: "29d4cc548b844b549739cf6394ce2f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" is_pre_election: true
I20260620 18:43:58.744704 17297 raft_consensus.cc:482] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:43:58.744731 17297 raft_consensus.cc:3037] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:58.744767 16941 raft_consensus.cc:2454] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29d4cc548b844b549739cf6394ce2f8b in term 0.
I20260620 18:43:58.745821 17297 raft_consensus.cc:504] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } }
I20260620 18:43:58.745958 17297 leader_election.cc:288] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [CANDIDATE]: Term 1 election: Requested vote from peers 8e33100310e04de685a859c19c07db02 (127.15.77.67:41011), acc3b0dff3e342538fe6bb04e8c79d0e (127.15.77.65:42065)
I20260620 18:43:58.746125 16941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8156699b4344c3eb1907d143ac76c53" candidate_uuid: "29d4cc548b844b549739cf6394ce2f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e"
I20260620 18:43:58.746116 17221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8156699b4344c3eb1907d143ac76c53" candidate_uuid: "29d4cc548b844b549739cf6394ce2f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e33100310e04de685a859c19c07db02"
I20260620 18:43:58.746196 16941 raft_consensus.cc:3037] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:58.746196 17221 raft_consensus.cc:3037] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:58.747018 16941 raft_consensus.cc:2454] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29d4cc548b844b549739cf6394ce2f8b in term 1.
I20260620 18:43:58.747018 17221 raft_consensus.cc:2454] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29d4cc548b844b549739cf6394ce2f8b in term 1.
I20260620 18:43:58.747195 17015 leader_election.cc:302] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29d4cc548b844b549739cf6394ce2f8b, acc3b0dff3e342538fe6bb04e8c79d0e; no voters:
I20260620 18:43:58.747299 17297 raft_consensus.cc:2781] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:43:58.747433 17297 raft_consensus.cc:686] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 1 LEADER]: Becoming Leader. State: Replica: 29d4cc548b844b549739cf6394ce2f8b, State: Running, Role: LEADER
I20260620 18:43:58.747520 17297 consensus_queue.cc:237] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } }
I20260620 18:43:58.748405 16784 catalog_manager.cc:5495] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 29d4cc548b844b549739cf6394ce2f8b (127.15.77.66). New cstate: current_term: 1 leader_uuid: "29d4cc548b844b549739cf6394ce2f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } health_report { overall_health: UNKNOWN } } }
I20260620 18:43:58.760797 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.757964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.15.77.66:36523)
0620 18:43:58.758046 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.758048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.758059 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:43:58.758171 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.758171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.758188 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.758248 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:43:58.758420 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.759203 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.760084 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.760270 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.760327 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.760494 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:43:58.760614 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:43:58.760676 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.760717 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.760736 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 18:43:58.762692 16941 raft_consensus.cc:1261] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e [term 1 FOLLOWER]: Refusing update from remote peer 29d4cc548b844b549739cf6394ce2f8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:43:58.762718 17221 raft_consensus.cc:1261] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02 [term 1 FOLLOWER]: Refusing update from remote peer 29d4cc548b844b549739cf6394ce2f8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:43:58.763101 17296 consensus_queue.cc:1035] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:43:58.763298 17297 consensus_queue.cc:1035] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 18:43:58.765257 17128 tablet.cc:2369] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:43:58.766100 17304 mvcc.cc:205] Tried to move back new op lower bound from 7298994334769102848 to 7298994334710673408. Current Snapshot: MvccSnapshot[applied={T|T < 7298994334769102848}]
I20260620 18:43:58.766151 17305 mvcc.cc:205] Tried to move back new op lower bound from 7298994334769102848 to 7298994334710673408. Current Snapshot: MvccSnapshot[applied={T|T < 7298994334769102848}]
I20260620 18:43:58.769946 16783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:43:58.770087 16783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:43:58.770905 16783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15: Sending DeleteTablet for 3 replicas of tablet f8156699b4344c3eb1907d143ac76c53
I20260620 18:43:58.771262 17061 tablet_service.cc:1514] Processing DeleteTablet for tablet f8156699b4344c3eb1907d143ac76c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:43:58 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:59586
I20260620 18:43:58.771381 16921 tablet_service.cc:1514] Processing DeleteTablet for tablet f8156699b4344c3eb1907d143ac76c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:43:58 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:44762
I20260620 18:43:58.771402 17201 tablet_service.cc:1514] Processing DeleteTablet for tablet f8156699b4344c3eb1907d143ac76c53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:43:58 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:57030
I20260620 18:43:58.771658 17314 tablet_replica.cc:330] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: stopping tablet replica
I20260620 18:43:58.771816 17314 raft_consensus.cc:2229] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:43:58.771818 16783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:43:58.771983 16783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:43:58.771991 17314 raft_consensus.cc:2258] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:43:58.771847 17312 tablet_replica.cc:330] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: stopping tablet replica
I20260620 18:43:58.772284 17312 raft_consensus.cc:2229] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:43:58.772460 17312 raft_consensus.cc:2258] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:43:58.772507 17314 ts_tablet_manager.cc:1905] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:43:58.772796 17313 tablet_replica.cc:330] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: stopping tablet replica
I20260620 18:43:58.772964 17313 raft_consensus.cc:2229] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:43:58.773185 17313 raft_consensus.cc:2258] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:43:58.773710 17313 ts_tablet_manager.cc:1905] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:43:58.773953 17061 tablet_service.cc:1467] Processing CreateTablet for tablet b4cda1c402e8420fa4ebb576cac576b3 (DEFAULT_TABLE table=test-table [id=e01ac07439d949128a44c07f0c247318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:43:58.774089 17061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4cda1c402e8420fa4ebb576cac576b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:58.774773 17314 ts_tablet_manager.cc:1918] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:43:58.774853 17314 log.cc:1196] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/f8156699b4344c3eb1907d143ac76c53
I20260620 18:43:58.775161 17314 ts_tablet_manager.cc:1939] T f8156699b4344c3eb1907d143ac76c53 P 8e33100310e04de685a859c19c07db02: Deleting consensus metadata
I20260620 18:43:58.775506 17290 tablet_bootstrap.cc:492] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b: Bootstrap starting.
I20260620 18:43:58.775750 16771 catalog_manager.cc:4841] TS 8e33100310e04de685a859c19c07db02 (127.15.77.67:41011): tablet f8156699b4344c3eb1907d143ac76c53 (table test-table [id=3dd5aaec308447cd9b64fae4ebc8bffd]) successfully deleted
I20260620 18:43:58.775826 16921 tablet_service.cc:1467] Processing CreateTablet for tablet b4cda1c402e8420fa4ebb576cac576b3 (DEFAULT_TABLE table=test-table [id=e01ac07439d949128a44c07f0c247318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:43:58.775936 16921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4cda1c402e8420fa4ebb576cac576b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:58.775962 17290 tablet_bootstrap.cc:654] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:58.775967 17201 tablet_service.cc:1467] Processing CreateTablet for tablet b4cda1c402e8420fa4ebb576cac576b3 (DEFAULT_TABLE table=test-table [id=e01ac07439d949128a44c07f0c247318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:43:58.776060 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4cda1c402e8420fa4ebb576cac576b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:58.776656 17313 ts_tablet_manager.cc:1918] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:43:58.776775 17313 log.cc:1196] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/f8156699b4344c3eb1907d143ac76c53
I20260620 18:43:58.777088 17313 ts_tablet_manager.cc:1939] T f8156699b4344c3eb1907d143ac76c53 P acc3b0dff3e342538fe6bb04e8c79d0e: Deleting consensus metadata
I20260620 18:43:58.777220 17291 tablet_bootstrap.cc:492] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: Bootstrap starting.
I20260620 18:43:58.777633 17291 tablet_bootstrap.cc:654] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:58.777900 17312 ts_tablet_manager.cc:1905] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:43:58.777982 17292 tablet_bootstrap.cc:492] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: Bootstrap starting.
I20260620 18:43:58.778337 17291 tablet_bootstrap.cc:492] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: No bootstrap required, opened a new log
I20260620 18:43:58.778388 17291 ts_tablet_manager.cc:1397] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:43:58.778395 17292 tablet_bootstrap.cc:654] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:58.778559 17291 raft_consensus.cc:348] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.778633 17291 raft_consensus.cc:374] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:58.778661 17291 raft_consensus.cc:729] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc3b0dff3e342538fe6bb04e8c79d0e, State: Initialized, Role: FOLLOWER
I20260620 18:43:58.778710 17291 consensus_queue.cc:260] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.778815 17291 ts_tablet_manager.cc:1428] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.778851 16771 catalog_manager.cc:4841] TS acc3b0dff3e342538fe6bb04e8c79d0e (127.15.77.65:42065): tablet f8156699b4344c3eb1907d143ac76c53 (table test-table [id=3dd5aaec308447cd9b64fae4ebc8bffd]) successfully deleted
I20260620 18:43:58.779255 17292 tablet_bootstrap.cc:492] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: No bootstrap required, opened a new log
I20260620 18:43:58.779299 17292 ts_tablet_manager.cc:1397] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 18:43:58.779502 17292 raft_consensus.cc:348] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.779580 17292 raft_consensus.cc:374] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:58.779603 17292 raft_consensus.cc:729] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e33100310e04de685a859c19c07db02, State: Initialized, Role: FOLLOWER
I20260620 18:43:58.779647 17292 consensus_queue.cc:260] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.779754 17292 ts_tablet_manager.cc:1428] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.779834 17312 ts_tablet_manager.cc:1918] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:43:58.779912 17312 log.cc:1196] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/f8156699b4344c3eb1907d143ac76c53
I20260620 18:43:58.780190 17312 ts_tablet_manager.cc:1939] T f8156699b4344c3eb1907d143ac76c53 P 29d4cc548b844b549739cf6394ce2f8b: Deleting consensus metadata
I20260620 18:43:58.780459 17290 tablet_bootstrap.cc:492] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b: No bootstrap required, opened a new log
I20260620 18:43:58.780550 17290 ts_tablet_manager.cc:1397] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260620 18:43:58.780603 16768 catalog_manager.cc:4841] TS 29d4cc548b844b549739cf6394ce2f8b (127.15.77.66:36523): tablet f8156699b4344c3eb1907d143ac76c53 (table test-table [id=3dd5aaec308447cd9b64fae4ebc8bffd]) successfully deleted
I20260620 18:43:58.780707 17290 raft_consensus.cc:348] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.780802 17290 raft_consensus.cc:374] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:58.780829 17290 raft_consensus.cc:729] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29d4cc548b844b549739cf6394ce2f8b, State: Initialized, Role: FOLLOWER
I20260620 18:43:58.780876 17290 consensus_queue.cc:260] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.780965 17290 ts_tablet_manager.cc:1428] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260620 18:43:58.861734 16988 tablet.cc:2369] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:43:58.916321 17297 raft_consensus.cc:482] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:43:58.916426 17297 raft_consensus.cc:504] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.916617 17297 leader_election.cc:288] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acc3b0dff3e342538fe6bb04e8c79d0e (127.15.77.65:42065), 8e33100310e04de685a859c19c07db02 (127.15.77.67:41011)
I20260620 18:43:58.916947 16941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4cda1c402e8420fa4ebb576cac576b3" candidate_uuid: "29d4cc548b844b549739cf6394ce2f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" is_pre_election: true
I20260620 18:43:58.916988 17221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4cda1c402e8420fa4ebb576cac576b3" candidate_uuid: "29d4cc548b844b549739cf6394ce2f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e33100310e04de685a859c19c07db02" is_pre_election: true
I20260620 18:43:58.917078 17221 raft_consensus.cc:2454] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29d4cc548b844b549739cf6394ce2f8b in term 0.
I20260620 18:43:58.917091 16941 raft_consensus.cc:2454] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29d4cc548b844b549739cf6394ce2f8b in term 0.
I20260620 18:43:58.917236 17015 leader_election.cc:302] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29d4cc548b844b549739cf6394ce2f8b, 8e33100310e04de685a859c19c07db02; no voters:
I20260620 18:43:58.917336 17297 raft_consensus.cc:2781] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:43:58.917390 17297 raft_consensus.cc:482] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:43:58.917421 17297 raft_consensus.cc:3037] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:58.918138 17297 raft_consensus.cc:504] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.918246 17297 leader_election.cc:288] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [CANDIDATE]: Term 1 election: Requested vote from peers acc3b0dff3e342538fe6bb04e8c79d0e (127.15.77.65:42065), 8e33100310e04de685a859c19c07db02 (127.15.77.67:41011)
I20260620 18:43:58.918462 16941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4cda1c402e8420fa4ebb576cac576b3" candidate_uuid: "29d4cc548b844b549739cf6394ce2f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e"
I20260620 18:43:58.918481 17221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4cda1c402e8420fa4ebb576cac576b3" candidate_uuid: "29d4cc548b844b549739cf6394ce2f8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e33100310e04de685a859c19c07db02"
I20260620 18:43:58.918540 16941 raft_consensus.cc:3037] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:58.918604 17221 raft_consensus.cc:3037] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:58.919180 16941 raft_consensus.cc:2454] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29d4cc548b844b549739cf6394ce2f8b in term 1.
I20260620 18:43:58.919374 17221 raft_consensus.cc:2454] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29d4cc548b844b549739cf6394ce2f8b in term 1.
I20260620 18:43:58.919394 17015 leader_election.cc:302] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29d4cc548b844b549739cf6394ce2f8b, acc3b0dff3e342538fe6bb04e8c79d0e; no voters:
I20260620 18:43:58.919493 17297 raft_consensus.cc:2781] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:43:58.919543 17297 raft_consensus.cc:686] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 1 LEADER]: Becoming Leader. State: Replica: 29d4cc548b844b549739cf6394ce2f8b, State: Running, Role: LEADER
I20260620 18:43:58.919610 17297 consensus_queue.cc:237] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.920302 16784 catalog_manager.cc:5495] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 29d4cc548b844b549739cf6394ce2f8b (127.15.77.66). New cstate: current_term: 1 leader_uuid: "29d4cc548b844b549739cf6394ce2f8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } health_report { overall_health: UNKNOWN } } }
W20260620 18:43:58.930799 17268 tablet.cc:2369] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 18:43:58 dist-test-slave-tvl9 krb5kdc[16739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:43:58.955279 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.948145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41739 (local address 127.0.0.1:37724)
0620 18:43:58.948355 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:58.948366 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:58.948430 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:58.948574 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:58.948575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:58.948884 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:43:58.948939 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.948940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.949687 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.949688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.950233 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:58.950235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.950585 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.950586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:58.950629 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.951148 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:43:58.951152 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:43:58.952642 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:43:58.954262 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.954270 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.954272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.954467 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.954669 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:43:58.954669 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:43:58.954670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:43:58.954729 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:43:58.955014 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:43:58.955020 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:43:58.955059 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 18:43:58.955112 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 18:43:58.955202 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":97,"threads_started":1}
I20260620 18:43:58.955441 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:58.948221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.15.77.126:41739)
0620 18:43:58.948290 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:58.948292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:58.948432 (+ 140us) server_negotiation.cc:407] Connection header received
0620 18:43:58.948481 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:58.948482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:58.948491 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:58.948555 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:58.949057 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.949591 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.950315 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:58.950516 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:58.950572 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:58.952778 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:58.952779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:58.952780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:58.952786 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:58.954185 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.954545 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.954546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.954546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.954608 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:58.954824 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:58.954826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:58.954826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:58.954943 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:58.955011 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:58.955156 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 18:43:58.955253 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":44}
I20260620 18:43:58.957508 16784 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:37724:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 18:43:58.957736 16784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:43:58.959239 16921 tablet_service.cc:1467] Processing CreateTablet for tablet a93dcead941a455d9eb4b2786ab26873 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3dffd9f859a2478688ad4c49af53a2ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 18:43:58.959242 17201 tablet_service.cc:1467] Processing CreateTablet for tablet a93dcead941a455d9eb4b2786ab26873 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3dffd9f859a2478688ad4c49af53a2ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 18:43:58.959242 17061 tablet_service.cc:1467] Processing CreateTablet for tablet a93dcead941a455d9eb4b2786ab26873 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3dffd9f859a2478688ad4c49af53a2ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 18:43:58.959419 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93dcead941a455d9eb4b2786ab26873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:58.959419 17061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93dcead941a455d9eb4b2786ab26873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:58.959429 16921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93dcead941a455d9eb4b2786ab26873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:43:58.960747 17291 tablet_bootstrap.cc:492] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: Bootstrap starting.
I20260620 18:43:58.960914 17290 tablet_bootstrap.cc:492] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: Bootstrap starting.
I20260620 18:43:58.961190 17291 tablet_bootstrap.cc:654] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:58.961221 17290 tablet_bootstrap.cc:654] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:58.961535 17292 tablet_bootstrap.cc:492] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: Bootstrap starting.
I20260620 18:43:58.961920 17290 tablet_bootstrap.cc:492] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: No bootstrap required, opened a new log
I20260620 18:43:58.961980 17290 ts_tablet_manager.cc:1397] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 18:43:58.961988 17292 tablet_bootstrap.cc:654] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: Neither blocks nor log segments found. Creating new log.
I20260620 18:43:58.962134 17290 raft_consensus.cc:348] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.962241 17290 raft_consensus.cc:374] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:58.962267 17290 raft_consensus.cc:729] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29d4cc548b844b549739cf6394ce2f8b, State: Initialized, Role: FOLLOWER
I20260620 18:43:58.962319 17290 consensus_queue.cc:260] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.962435 17296 tablet_replica.cc:439] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } }
I20260620 18:43:58.962598 17296 tablet_replica.cc:442] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:43:58.962695 17291 tablet_bootstrap.cc:492] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: No bootstrap required, opened a new log
I20260620 18:43:58.962716 17292 tablet_bootstrap.cc:492] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: No bootstrap required, opened a new log
I20260620 18:43:58.962752 17292 ts_tablet_manager.cc:1397] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:43:58.962751 17291 ts_tablet_manager.cc:1397] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:43:58.962899 17292 raft_consensus.cc:348] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.962904 17291 raft_consensus.cc:348] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.962972 17291 raft_consensus.cc:374] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:58.962972 17292 raft_consensus.cc:374] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:43:58.962994 17292 raft_consensus.cc:729] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e33100310e04de685a859c19c07db02, State: Initialized, Role: FOLLOWER
I20260620 18:43:58.962994 17291 raft_consensus.cc:729] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc3b0dff3e342538fe6bb04e8c79d0e, State: Initialized, Role: FOLLOWER
I20260620 18:43:58.963037 17292 consensus_queue.cc:260] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.963037 17291 consensus_queue.cc:260] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:58.963142 17292 ts_tablet_manager.cc:1428] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.963155 17291 ts_tablet_manager.cc:1428] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:43:58.963193 17300 tablet_replica.cc:439] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } }
I20260620 18:43:58.963424 17298 tablet_replica.cc:439] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } }
I20260620 18:43:58.962436 17290 ts_tablet_manager.cc:1428] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 18:43:58.963521 17298 tablet_replica.cc:442] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:43:58.963296 17300 tablet_replica.cc:442] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:43:59.046386 17298 raft_consensus.cc:482] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:43:59.046518 17298 raft_consensus.cc:504] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:59.046837 17298 leader_election.cc:288] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acc3b0dff3e342538fe6bb04e8c79d0e (127.15.77.65:42065), 29d4cc548b844b549739cf6394ce2f8b (127.15.77.66:36523)
I20260620 18:43:59.049834 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.047132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:39001 (local address 127.15.77.66:36523)
0620 18:43:59.047184 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:59.047186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:59.047229 (+ 43us) server_negotiation.cc:407] Connection header received
0620 18:43:59.047259 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:59.047260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:59.047275 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:59.047410 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:59.047587 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.048081 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.049050 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.049421 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.049528 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.049539 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:59.049710 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 18:43:59.049760 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:43:59.049845 17160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.047038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:36523 (local address 127.15.77.67:39001)
0620 18:43:59.047080 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:59.047093 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:59.047158 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:59.047421 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:59.047422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:59.047424 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:59.047499 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:59.047501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.048174 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.048175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:59.048936 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:59.048938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.049530 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.049531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:59.049624 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.049628 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:43:59.049702 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 18:43:59.049744 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260620 18:43:59.050199 17081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93dcead941a455d9eb4b2786ab26873" candidate_uuid: "8e33100310e04de685a859c19c07db02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29d4cc548b844b549739cf6394ce2f8b" is_pre_election: true
I20260620 18:43:59.050338 17081 raft_consensus.cc:2454] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e33100310e04de685a859c19c07db02 in term 0.
I20260620 18:43:59.050551 17152 leader_election.cc:302] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29d4cc548b844b549739cf6394ce2f8b, 8e33100310e04de685a859c19c07db02; no voters:
I20260620 18:43:59.050681 17298 raft_consensus.cc:2781] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:43:59.050722 17298 raft_consensus.cc:482] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:43:59.050776 17298 raft_consensus.cc:3037] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:59.051551 17298 raft_consensus.cc:504] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:59.051704 17298 leader_election.cc:288] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [CANDIDATE]: Term 1 election: Requested vote from peers acc3b0dff3e342538fe6bb04e8c79d0e (127.15.77.65:42065), 29d4cc548b844b549739cf6394ce2f8b (127.15.77.66:36523)
I20260620 18:43:59.051872 17081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93dcead941a455d9eb4b2786ab26873" candidate_uuid: "8e33100310e04de685a859c19c07db02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29d4cc548b844b549739cf6394ce2f8b"
I20260620 18:43:59.051997 17081 raft_consensus.cc:3037] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:59.052042 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.047129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:53013 (local address 127.15.77.65:42065)
0620 18:43:59.047177 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:59.047179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:59.047377 (+ 198us) server_negotiation.cc:407] Connection header received
0620 18:43:59.047433 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:59.047435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:59.047444 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:59.047526 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:43:59.047708 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.048189 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.051180 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.051615 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.051730 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.051740 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:59.051930 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 18:43:59.051977 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:43:59.052063 17328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.047038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:42065 (local address 127.15.77.67:53013)
0620 18:43:59.047292 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:59.047299 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:59.047381 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:59.047532 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:59.047533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:59.047535 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:43:59.047630 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:59.047631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.048277 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.048278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:59.051073 (+ 2795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:59.051076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.051760 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.051761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:59.051834 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.051837 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:43:59.051923 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 18:43:59.051965 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":71,"thread_start_us":114,"threads_started":1}
I20260620 18:43:59.052337 16941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93dcead941a455d9eb4b2786ab26873" candidate_uuid: "8e33100310e04de685a859c19c07db02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" is_pre_election: true
I20260620 18:43:59.052416 16941 raft_consensus.cc:2454] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e33100310e04de685a859c19c07db02 in term 0.
I20260620 18:43:59.052368 16940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93dcead941a455d9eb4b2786ab26873" candidate_uuid: "8e33100310e04de685a859c19c07db02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e"
I20260620 18:43:59.052455 16940 raft_consensus.cc:3037] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:43:59.052927 17081 raft_consensus.cc:2454] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e33100310e04de685a859c19c07db02 in term 1.
I20260620 18:43:59.053148 17152 leader_election.cc:302] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29d4cc548b844b549739cf6394ce2f8b, 8e33100310e04de685a859c19c07db02; no voters:
I20260620 18:43:59.053170 16940 raft_consensus.cc:2454] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e33100310e04de685a859c19c07db02 in term 1.
I20260620 18:43:59.053267 17298 raft_consensus.cc:2781] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:43:59.053448 17298 raft_consensus.cc:686] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [term 1 LEADER]: Becoming Leader. State: Replica: 8e33100310e04de685a859c19c07db02, State: Running, Role: LEADER
I20260620 18:43:59.053553 17298 consensus_queue.cc:237] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } }
I20260620 18:43:59.053915 17329 tablet_replica.cc:439] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: TxnStatusTablet state changed. Reason: New leader 8e33100310e04de685a859c19c07db02. Latest consensus state: current_term: 1 leader_uuid: "8e33100310e04de685a859c19c07db02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } }
I20260620 18:43:59.053998 17329 tablet_replica.cc:442] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: This TxnStatusTablet replica's current role is: LEADER
I20260620 18:43:59.054196 17331 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 18:43:59.054323 17331 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 18:43:59.054349 16782 catalog_manager.cc:5495] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e33100310e04de685a859c19c07db02 (127.15.77.67). New cstate: current_term: 1 leader_uuid: "8e33100310e04de685a859c19c07db02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } health_report { overall_health: HEALTHY } } }
I20260620 18:43:59.080811 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.078135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:41011 (local address 127.0.0.1:57034)
0620 18:43:59.078222 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:59.078251 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:59.078302 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:59.078511 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:59.078512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:59.078513 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:43:59.078564 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:59.078567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.079200 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.079202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:59.079869 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:59.079871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.080244 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.080245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:59.080315 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.080332 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:43:59.080629 (+ 297us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:43:59.080639 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 18:43:59.080695 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 18:43:59.080743 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 18:43:59.080802 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.078207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57034 (local address 127.15.77.67:41011)
0620 18:43:59.078274 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:59.078276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:59.078286 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:43:59.078378 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:59.078379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:59.078400 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:59.078498 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:43:59.078643 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.079114 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.079961 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.080161 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.080261 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.080452 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:43:59.080560 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:43:59.080644 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:59.080701 (+ 57us) server_negotiation.cc:299] Negotiation successful
0620 18:43:59.080721 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 18:43:59.082054 16940 raft_consensus.cc:1261] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e [term 1 FOLLOWER]: Refusing update from remote peer 8e33100310e04de685a859c19c07db02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:43:59.082135 17081 raft_consensus.cc:1261] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b [term 1 FOLLOWER]: Refusing update from remote peer 8e33100310e04de685a859c19c07db02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:43:59.082404 17298 consensus_queue.cc:1035] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:43:59.082648 17329 consensus_queue.cc:1035] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:43:59.082988 17300 tablet_replica.cc:439] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: TxnStatusTablet state changed. Reason: New leader 8e33100310e04de685a859c19c07db02. Latest consensus state: current_term: 1 leader_uuid: "8e33100310e04de685a859c19c07db02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } }
I20260620 18:43:59.083081 17300 tablet_replica.cc:442] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:43:59.083266 17296 tablet_replica.cc:439] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: TxnStatusTablet state changed. Reason: New leader 8e33100310e04de685a859c19c07db02. Latest consensus state: current_term: 1 leader_uuid: "8e33100310e04de685a859c19c07db02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } }
I20260620 18:43:59.083375 17296 tablet_replica.cc:442] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:43:59.083714 17329 tablet_replica.cc:439] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e33100310e04de685a859c19c07db02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } }
I20260620 18:43:59.083791 17329 tablet_replica.cc:442] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: This TxnStatusTablet replica's current role is: LEADER
I20260620 18:43:59.084218 17300 tablet_replica.cc:439] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e33100310e04de685a859c19c07db02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } }
I20260620 18:43:59.084292 17300 tablet_replica.cc:442] T a93dcead941a455d9eb4b2786ab26873 P acc3b0dff3e342538fe6bb04e8c79d0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:43:59.085275 17298 tablet_replica.cc:439] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e33100310e04de685a859c19c07db02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } }
I20260620 18:43:59.085383 17298 tablet_replica.cc:442] T a93dcead941a455d9eb4b2786ab26873 P 8e33100310e04de685a859c19c07db02: This TxnStatusTablet replica's current role is: LEADER
I20260620 18:43:59.085601 17296 tablet_replica.cc:439] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e33100310e04de685a859c19c07db02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 } } peers { permanent_uuid: "29d4cc548b844b549739cf6394ce2f8b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36523 } } peers { permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 } } }
I20260620 18:43:59.085695 17296 tablet_replica.cc:442] T a93dcead941a455d9eb4b2786ab26873 P 29d4cc548b844b549739cf6394ce2f8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:43:59.091080 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.088284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:41011 (local address 127.15.77.66:50943)
0620 18:43:59.088509 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:59.088521 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:59.088596 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:59.088748 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:59.088749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:59.088750 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:43:59.088793 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:59.088794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.089665 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.089666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:59.090242 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:59.090243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.090718 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.090719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:59.090762 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.090777 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:43:59.090948 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:43:59.090949 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 18:43:59.090984 (+ 35us) client_negotiation.cc:240] Negotiation successful
0620 18:43:59.091022 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":58,"threads_started":1}
I20260620 18:43:59.091082 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.088351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:50943 (local address 127.15.77.67:41011)
0620 18:43:59.088395 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:59.088397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:59.088582 (+ 185us) server_negotiation.cc:407] Connection header received
0620 18:43:59.088661 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:59.088662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:59.088672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:59.088738 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:43:59.088848 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.089585 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.090340 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.090645 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.090714 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.090854 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:43:59.090888 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:43:59.090943 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:59.090987 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 18:43:59.091007 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:43:59.093583 17221 raft_consensus.cc:1261] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02 [term 1 FOLLOWER]: Refusing update from remote peer 29d4cc548b844b549739cf6394ce2f8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:43:59.093577 16940 raft_consensus.cc:1261] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e [term 1 FOLLOWER]: Refusing update from remote peer 29d4cc548b844b549739cf6394ce2f8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:43:59.093792 17296 consensus_queue.cc:1035] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e33100310e04de685a859c19c07db02" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:43:59.093914 17297 consensus_queue.cc:1035] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "acc3b0dff3e342538fe6bb04e8c79d0e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:43:59.095130 17303 mvcc.cc:205] Tried to move back new op lower bound from 7298994336125878272 to 7298994335415185408. Current Snapshot: MvccSnapshot[applied={T|T < 7298994336125878272}]
I20260620 18:43:59.102663 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.100309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:44477 (local address 127.15.77.66:36523)
0620 18:43:59.100369 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:59.100371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:59.100377 (+ 6us) server_negotiation.cc:407] Connection header received
0620 18:43:59.100397 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:59.100398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:59.100408 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:59.100479 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:43:59.100592 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.101074 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.101910 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.102199 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.102258 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.102460 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:43:59.102489 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:43:59.102529 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:59.102585 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 18:43:59.102607 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 18:43:59.102701 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.100133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:36523 (local address 127.15.77.67:44477)
0620 18:43:59.100206 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 18:43:59.100223 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 18:43:59.100254 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:43:59.100483 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:43:59.100483 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:43:59.100485 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:43:59.100535 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:59.100537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.101156 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.101157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:59.101821 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:43:59.101823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.102276 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.102277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:43:59.102357 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.102377 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:43:59.102534 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:43:59.102536 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:43:59.102576 (+ 40us) client_negotiation.cc:240] Negotiation successful
0620 18:43:59.102621 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 18:43:59.112684 16783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:43:59.112818 16783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:43:59.113485 16783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f00d5c7713d44075b03dd5e338b42e15: Sending DeleteTablet for 3 replicas of tablet b4cda1c402e8420fa4ebb576cac576b3
I20260620 18:43:59.113749 16921 tablet_service.cc:1514] Processing DeleteTablet for tablet b4cda1c402e8420fa4ebb576cac576b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:43:59 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:44762
I20260620 18:43:59.113854 17313 tablet_replica.cc:330] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: stopping tablet replica
I20260620 18:43:59.113821 17061 tablet_service.cc:1514] Processing DeleteTablet for tablet b4cda1c402e8420fa4ebb576cac576b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:43:59 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:59586
I20260620 18:43:59.113897 17313 raft_consensus.cc:2229] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:43:59.113950 17313 raft_consensus.cc:2258] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:43:59.113936 17312 tablet_replica.cc:330] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b: stopping tablet replica
I20260620 18:43:59.114003 17312 raft_consensus.cc:2229] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:43:59.114076 17312 raft_consensus.cc:2258] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:43:59.114050 17201 tablet_service.cc:1514] Processing DeleteTablet for tablet b4cda1c402e8420fa4ebb576cac576b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:43:59 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:57030
I20260620 18:43:59.114209 17314 tablet_replica.cc:330] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: stopping tablet replica
I20260620 18:43:59.114280 17313 ts_tablet_manager.cc:1905] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:43:59.114289 17314 raft_consensus.cc:2229] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:43:59.114342 17314 raft_consensus.cc:2258] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:43:59.114696 17314 ts_tablet_manager.cc:1905] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:43:59.116091 17313 ts_tablet_manager.cc:1918] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:43:59.116128 17314 ts_tablet_manager.cc:1918] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:43:59.116154 17313 log.cc:1196] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/b4cda1c402e8420fa4ebb576cac576b3
I20260620 18:43:59.116156 17312 ts_tablet_manager.cc:1905] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:43:59.116181 17314 log.cc:1196] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/b4cda1c402e8420fa4ebb576cac576b3
I20260620 18:43:59.116379 17313 ts_tablet_manager.cc:1939] T b4cda1c402e8420fa4ebb576cac576b3 P acc3b0dff3e342538fe6bb04e8c79d0e: Deleting consensus metadata
I20260620 18:43:59.116407 17314 ts_tablet_manager.cc:1939] T b4cda1c402e8420fa4ebb576cac576b3 P 8e33100310e04de685a859c19c07db02: Deleting consensus metadata
I20260620 18:43:59.116751 16771 catalog_manager.cc:4841] TS acc3b0dff3e342538fe6bb04e8c79d0e (127.15.77.65:42065): tablet b4cda1c402e8420fa4ebb576cac576b3 (table test-table [id=e01ac07439d949128a44c07f0c247318]) successfully deleted
I20260620 18:43:59.117578 17312 ts_tablet_manager.cc:1918] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:43:59.117607 16771 catalog_manager.cc:4841] TS 8e33100310e04de685a859c19c07db02 (127.15.77.67:41011): tablet b4cda1c402e8420fa4ebb576cac576b3 (table test-table [id=e01ac07439d949128a44c07f0c247318]) successfully deleted
I20260620 18:43:59.117636 17312 log.cc:1196] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/b4cda1c402e8420fa4ebb576cac576b3
I20260620 18:43:59.117869 17312 ts_tablet_manager.cc:1939] T b4cda1c402e8420fa4ebb576cac576b3 P 29d4cc548b844b549739cf6394ce2f8b: Deleting consensus metadata
I20260620 18:43:59.118356 16768 catalog_manager.cc:4841] TS 29d4cc548b844b549739cf6394ce2f8b (127.15.77.66:36523): tablet b4cda1c402e8420fa4ebb576cac576b3 (table test-table [id=e01ac07439d949128a44c07f0c247318]) successfully deleted
Jun 20 18:43:59 dist-test-slave-tvl9 krb5kdc[16739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981038, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:43:59.128194 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.118476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.15.77.65:42065)
0620 18:43:59.118520 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:59.118522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:59.118651 (+ 129us) server_negotiation.cc:407] Connection header received
0620 18:43:59.118682 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:59.118683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:59.118696 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:59.118766 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:59.119466 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.120312 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.121059 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.121369 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.121439 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.124670 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:59.124684 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:59.124684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:59.124693 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:59.126898 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:59.127265 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:59.127269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:59.127270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:59.127397 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:59.127619 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:59.127620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:59.127620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:59.127818 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:59.127972 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:59.128015 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 18:43:59.128101 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 18:43:59.128432 16981 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44772
I20260620 18:43:59.135987 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:43:59.129467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37726 (local address 127.15.77.126:41739)
0620 18:43:59.129529 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 18:43:59.129531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:43:59.129541 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:43:59.129566 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:43:59.129567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:43:59.129579 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:43:59.129655 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:43:59.130160 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.130942 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.131676 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:43:59.131970 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:43:59.132052 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:43:59.133203 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:43:59.133205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:43:59.133205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:43:59.133213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:43:59.134794 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:59.135109 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:59.135110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:59.135111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:59.135174 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:43:59.135406 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:43:59.135407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:43:59.135407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:43:59.135549 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:43:59.135648 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 18:43:59.135812 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 18:43:59.135892 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260620 18:43:59.136195 16783 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37726
I20260620 18:43:59.136881 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 16856
I20260620 18:43:59.143635 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 16996
I20260620 18:43:59.150460 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 17136
I20260620 18:43:59.155601 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 16752
2026-06-20T18:43:59Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2275 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:43:59 dist-test-slave-tvl9 krb5kdc[17351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:43:59 dist-test-slave-tvl9 krb5kdc[17351](info): set up 2 sockets
Jun 20 18:43:59 dist-test-slave-tvl9 krb5kdc[17351](info): commencing operation
krb5kdc: starting...
W20260620 18:44:00.199019 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:44:00 dist-test-slave-tvl9 krb5kdc[17351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:44:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:44:00.322613 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:36545
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:36545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:00.404016 17364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:00.404223 17364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:00.404245 17364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:00.405324 17364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:00.405359 17364 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 18:44:00.405372 17364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:00.405385 17364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:00.405396 17364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:00.405408 17364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:00.406836 17364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:36545
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:36545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:00.407038 17364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:00.407214 17364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:00.412132 17372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:00.412143 17369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:00.412213 17364 server_base.cc:1034] running on GCE node
W20260620 18:44:00.412343 17370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:00.412616 17364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:00.412928 17364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:00.414088 17364 hybrid_clock.cc:648] HybridClock initialized: now 1781981040414067 us; error 37 us; skew 500 ppm
Jun 20 18:44:00 dist-test-slave-tvl9 krb5kdc[17351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:00.416790 17364 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:44:00.417338 17364 webserver.cc:460] Webserver started at http://127.15.77.126:46649/ using document root <none> and password file <none>
I20260620 18:44:00.417590 17364 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:00.417660 17364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:00.417812 17364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:00.418898 17364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "9cc296fac101455da43510587f70c454"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "f87738ed13de0d0776bf7cf2a3c5463a"
server_key_iv: "d766313a0bf447f8029e92e3b8dfe5d6"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.419272 17364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "9cc296fac101455da43510587f70c454"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "f87738ed13de0d0776bf7cf2a3c5463a"
server_key_iv: "d766313a0bf447f8029e92e3b8dfe5d6"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.420747 17364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:00.421605 17379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:00.421790 17364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:00.421895 17364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "9cc296fac101455da43510587f70c454"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "f87738ed13de0d0776bf7cf2a3c5463a"
server_key_iv: "d766313a0bf447f8029e92e3b8dfe5d6"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.421975 17364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:00.454936 17364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:00.457196 17364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:00.457376 17364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:00.462360 17364 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:36545
I20260620 18:44:00.462517 17441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:36545 every 8 connection(s)
I20260620 18:44:00.462831 17364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:00.463683 17442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:00.466022 17442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454: Bootstrap starting.
I20260620 18:44:00.466651 17442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:00.466964 17442 log.cc:824] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:00.467711 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 17364
I20260620 18:44:00.467754 17442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454: No bootstrap required, opened a new log
I20260620 18:44:00.467815 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:00.467967 15669 external_mini_cluster.cc:1428] Setting key d25d12c739f4272d5c9556d889ef6c10
I20260620 18:44:00.471928 17442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc296fac101455da43510587f70c454" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36545 } }
I20260620 18:44:00.472123 17442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:00.472157 17442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cc296fac101455da43510587f70c454, State: Initialized, Role: FOLLOWER
I20260620 18:44:00.472278 17442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc296fac101455da43510587f70c454" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36545 } }
I20260620 18:44:00.472352 17442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:00.472381 17442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:00.472426 17442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:00.473212 17442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc296fac101455da43510587f70c454" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36545 } }
I20260620 18:44:00.473305 17442 leader_election.cc:302] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cc296fac101455da43510587f70c454; no voters:
I20260620 18:44:00.473512 17442 leader_election.cc:288] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:00.473552 17447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:00.473659 17447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [term 1 LEADER]: Becoming Leader. State: Replica: 9cc296fac101455da43510587f70c454, State: Running, Role: LEADER
I20260620 18:44:00.473739 17447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc296fac101455da43510587f70c454" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36545 } }
I20260620 18:44:00.473784 17442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:00.474068 17447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cc296fac101455da43510587f70c454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc296fac101455da43510587f70c454" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36545 } } }
I20260620 18:44:00.474143 17447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:00.474272 17447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cc296fac101455da43510587f70c454. Latest consensus state: current_term: 1 leader_uuid: "9cc296fac101455da43510587f70c454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc296fac101455da43510587f70c454" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36545 } } }
I20260620 18:44:00.474340 17447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454 [sys.catalog]: This master's current role is: LEADER
Jun 20 18:44:00 dist-test-slave-tvl9 krb5kdc[17351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
W20260620 18:44:00.475613 17463 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:44:00.475669 17463 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:44:00.475714 17452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:00.475906 17452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:00.477473 17452 catalog_manager.cc:1347] Generated new cluster ID: 193ed479a6884baf9e66afa16f914316
I20260620 18:44:00.477524 17452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:44:00.479334 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.468538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43512 (local address 127.15.77.126:36545)
0620 18:44:00.468724 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:00.468734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:00.468769 (+ 35us) server_negotiation.cc:407] Connection header received
0620 18:44:00.469263 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:00.469272 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:00.469340 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:00.469506 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:00.470094 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.471062 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.471858 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.472465 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.472585 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.475557 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:00.475562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:00.475562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:00.475573 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:00.477609 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.477995 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.477998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.477998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.478107 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.478313 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.478314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.478314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.478644 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:00.478801 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:00.478922 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 18:44:00.479063 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260620 18:44:00.491614 17452 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:00.492262 17452 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:00.495836 17452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454: Generated new TSK 0
I20260620 18:44:00.496052 17452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:44:00.521893 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:36545
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:00.604849 17468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:00.605087 17468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:00.605114 17468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:00.606367 17468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:00.606410 17468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:00.606438 17468 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 18:44:00.606451 17468 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 18:44:00.606467 17468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:44:00.607949 17468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.77.126:36545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:00.608151 17468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:00.608381 17468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:00.613538 17474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:00.613539 17476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:00.613538 17473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:00.613758 17468 server_base.cc:1034] running on GCE node
I20260620 18:44:00.613974 17468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:00.614257 17468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:00.615381 17468 hybrid_clock.cc:648] HybridClock initialized: now 1781981040615377 us; error 38 us; skew 500 ppm
Jun 20 18:44:00 dist-test-slave-tvl9 krb5kdc[17351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:00.618142 17468 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:44:00.618640 17468 webserver.cc:460] Webserver started at http://127.15.77.65:41665/ using document root <none> and password file <none>
I20260620 18:44:00.618872 17468 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:00.618919 17468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:00.619055 17468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:00.620219 17468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "f99b7f4a80a8471d885cb4652fefdc6c"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "b734f6fd97939b0a1836966e1f2712a2"
server_key_iv: "c2abe7a2d39b93f0c5ca569dbcf2b78b"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.620585 17468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "f99b7f4a80a8471d885cb4652fefdc6c"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "b734f6fd97939b0a1836966e1f2712a2"
server_key_iv: "c2abe7a2d39b93f0c5ca569dbcf2b78b"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.621971 17468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:44:00.622714 17483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:00.622903 17468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:44:00.622993 17468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "f99b7f4a80a8471d885cb4652fefdc6c"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "b734f6fd97939b0a1836966e1f2712a2"
server_key_iv: "c2abe7a2d39b93f0c5ca569dbcf2b78b"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.623064 17468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:00.651780 17468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:00.654299 17468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:00.654414 17468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:00.654897 17468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:00.654935 17468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:00.654963 17468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:00.655015 17468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:00.661895 17468 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:37547
I20260620 18:44:00.661948 17598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:37547 every 8 connection(s)
I20260620 18:44:00.662319 17468 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
Jun 20 18:44:00 dist-test-slave-tvl9 krb5kdc[17351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:00.665403 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.655528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:51943 (local address 127.15.77.126:36545)
0620 18:44:00.655571 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:00.655573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:00.655849 (+ 276us) server_negotiation.cc:407] Connection header received
0620 18:44:00.656477 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:00.656479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:00.656494 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:00.656578 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:00.657449 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.658322 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.659970 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.660303 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.660394 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.662999 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:00.663001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:00.663001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:00.663007 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:00.664275 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.664650 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.664650 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.664651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.664702 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.664901 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.664902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.664902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.664999 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:00.665051 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:00.665268 (+ 217us) server_negotiation.cc:299] Negotiation successful
0620 18:44:00.665338 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:44:00.665589 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.655531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36545 (local address 127.15.77.65:51943)
0620 18:44:00.655716 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:00.655752 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:00.656317 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:00.656653 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:00.656658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:00.657020 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:00.657332 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.657339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.658432 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.658434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.659878 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.659882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.660396 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.660397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.660537 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.661134 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:00.661145 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:00.662918 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:00.664342 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.664348 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.664350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.664586 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.664757 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.664758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.664758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.664851 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.665052 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:00.665056 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:00.665163 (+ 107us) client_negotiation.cc:769] Sending connection context
0620 18:44:00.665243 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 18:44:00.665358 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
I20260620 18:44:00.666160 17599 heartbeater.cc:347] Connected to a master server at 127.15.77.126:36545
I20260620 18:44:00.666251 17599 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:00.666448 17599 heartbeater.cc:511] Master 127.15.77.126:36545 requested a full tablet report, sending...
I20260620 18:44:00.667002 17396 ts_manager.cc:194] Registered new tserver with Master: f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65:37547)
I20260620 18:44:00.667126 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 17468
I20260620 18:44:00.667228 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:44:00.667471 15669 external_mini_cluster.cc:1428] Setting key 9d1edcd7bdb9b120321cbc44350d3888
I20260620 18:44:00.668022 17396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:51943
I20260620 18:44:00.673676 17605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.667269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36545 (local address 127.15.77.65:36797)
0620 18:44:00.667436 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:00.667447 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:00.667505 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:00.667639 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:00.667639 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:00.667830 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:00.667889 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.667891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.668616 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.668617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.669436 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.669438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.669924 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.669926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.670015 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.670398 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:00.670404 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:00.670841 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:00.672690 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.672691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.672691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.672850 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.673064 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.673064 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.673065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.673115 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.673388 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:00.673388 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:00.673421 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 18:44:00.673485 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 18:44:00.673588 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260620 18:44:00.673761 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.667327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:36797 (local address 127.15.77.126:36545)
0620 18:44:00.667411 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:00.667413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:00.667544 (+ 131us) server_negotiation.cc:407] Connection header received
0620 18:44:00.667565 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:00.667565 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:00.667576 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:00.667634 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:00.667960 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.668530 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.669520 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.669837 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.669913 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.670941 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:00.670943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:00.670943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:00.670948 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:00.672599 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.672924 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.672925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.672926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.672987 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.673175 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.673176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.673177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.673322 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:00.673395 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:00.673521 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 18:44:00.673623 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:44:00.694327 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:36545
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:00.776849 17608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:00.777067 17608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:00.777096 17608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:00.778259 17608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:00.778303 17608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:00.778324 17608 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 18:44:00.778337 17608 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 18:44:00.778354 17608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:44:00.779768 17608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.77.126:36545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:00.779951 17608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:00.780138 17608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:00.785090 17616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:00.785128 17613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:00.785125 17614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:00.785252 17608 server_base.cc:1034] running on GCE node
I20260620 18:44:00.785740 17608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:00.785990 17608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:00.787186 17608 hybrid_clock.cc:648] HybridClock initialized: now 1781981040787178 us; error 44 us; skew 500 ppm
Jun 20 18:44:00 dist-test-slave-tvl9 krb5kdc[17351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:00.789706 17608 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:44:00.790179 17608 webserver.cc:460] Webserver started at http://127.15.77.66:40945/ using document root <none> and password file <none>
I20260620 18:44:00.790434 17608 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:00.790503 17608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:00.790658 17608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:00.791785 17608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "f84d7b54d43948dbb4634a64af7bbd2b"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "f81b61454102a0a9ebd30e5797bce3ac"
server_key_iv: "26604adf8af89a96ae2f44dddd5c681b"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.792157 17608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "f84d7b54d43948dbb4634a64af7bbd2b"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "f81b61454102a0a9ebd30e5797bce3ac"
server_key_iv: "26604adf8af89a96ae2f44dddd5c681b"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.793550 17608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260620 18:44:00.794279 17623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:00.794456 17608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:00.794546 17608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "f84d7b54d43948dbb4634a64af7bbd2b"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "f81b61454102a0a9ebd30e5797bce3ac"
server_key_iv: "26604adf8af89a96ae2f44dddd5c681b"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.794631 17608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:00.807969 17608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:00.810422 17608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:00.810595 17608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:00.811141 17608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:00.811201 17608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:00.811235 17608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:00.811273 17608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 18:44:00 dist-test-slave-tvl9 krb5kdc[17351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:00.818871 17608 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:37075
I20260620 18:44:00.819243 17738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:37075 every 8 connection(s)
I20260620 18:44:00.819303 17608 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:44:00.821867 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.811833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:54993 (local address 127.15.77.126:36545)
0620 18:44:00.811897 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:00.811899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:00.812125 (+ 226us) server_negotiation.cc:407] Connection header received
0620 18:44:00.812649 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:00.812651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:00.812667 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:00.812745 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:00.813341 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.814189 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.815301 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.815554 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.815646 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.818915 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:00.818916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:00.818917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:00.818924 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:00.820453 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.820884 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.820885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.820886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.820951 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.821212 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.821213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.821214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.821339 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:00.821415 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:00.821689 (+ 274us) server_negotiation.cc:299] Negotiation successful
0620 18:44:00.821774 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 18:44:00.822073 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.811827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36545 (local address 127.15.77.66:54993)
0620 18:44:00.812011 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:00.812048 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:00.812543 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:00.812803 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:00.812806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:00.813041 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:00.813233 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.813237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.814290 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.814291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.815205 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.815208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.815657 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.815658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.815794 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.816356 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:00.816368 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:00.818805 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:00.820554 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.820562 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.820564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.820805 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.821017 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.821017 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.821018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.821137 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.821405 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:00.821412 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:00.821558 (+ 146us) client_negotiation.cc:769] Sending connection context
0620 18:44:00.821668 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 18:44:00.821832 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260620 18:44:00.822570 17739 heartbeater.cc:347] Connected to a master server at 127.15.77.126:36545
I20260620 18:44:00.822650 17739 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:00.822834 17739 heartbeater.cc:511] Master 127.15.77.126:36545 requested a full tablet report, sending...
I20260620 18:44:00.823238 17396 ts_manager.cc:194] Registered new tserver with Master: f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075)
I20260620 18:44:00.823709 17396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:54993
I20260620 18:44:00.829252 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 17608
I20260620 18:44:00.829291 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.823784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:57925 (local address 127.15.77.126:36545)
0620 18:44:00.823826 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:00.823828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:00.823943 (+ 115us) server_negotiation.cc:407] Connection header received
0620 18:44:00.824014 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:00.824014 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:00.824024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:00.824083 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:00.824418 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.824962 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.825736 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.825937 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.826008 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.826861 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:00.826863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:00.826863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:00.826870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:00.828090 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.828433 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.828434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.828434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.828492 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.828696 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.828697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.828698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.828868 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:00.828944 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:00.829125 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 18:44:00.829200 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:44:00.829294 17745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.823725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36545 (local address 127.15.77.66:57925)
0620 18:44:00.823891 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:00.823903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:00.823958 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:00.824101 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:00.824102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:00.824286 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:00.824342 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.824343 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.825054 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.825055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.825629 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.825630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.826007 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.826008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.826081 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.826478 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:00.826483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:00.826771 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:00.828173 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.828174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.828175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.828347 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.828570 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.828571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.828571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.828633 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.828947 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:00.828949 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:00.828995 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 18:44:00.829085 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 18:44:00.829194 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260620 18:44:00.829362 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:44:00.829584 15669 external_mini_cluster.cc:1428] Setting key d2314b6f6b288a83c1f9247dbd96c986
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:44:00.854772 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:36545
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:00.938583 17748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:00.938802 17748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:00.938828 17748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:00.940104 17748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:00.940148 17748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:00.940169 17748 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 18:44:00.940186 17748 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 18:44:00.940203 17748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:44:00.941689 17748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.77.126:36545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:00.941937 17748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:00.942189 17748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:00.947300 17756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:00.947335 17753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:00.947335 17754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:00.947486 17748 server_base.cc:1034] running on GCE node
I20260620 18:44:00.947757 17748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:00.948022 17748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:00.949184 17748 hybrid_clock.cc:648] HybridClock initialized: now 1781981040949168 us; error 43 us; skew 500 ppm
Jun 20 18:44:00 dist-test-slave-tvl9 krb5kdc[17351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:00.951574 17748 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:44:00.952054 17748 webserver.cc:460] Webserver started at http://127.15.77.67:41873/ using document root <none> and password file <none>
I20260620 18:44:00.952303 17748 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:00.952373 17748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:00.952523 17748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:00.953569 17748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "70753283514f4381bed7e3f3b3e8bbad"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "02753e44c7ef2a42843dae76839c6e70"
server_key_iv: "0697bce9e36fa595ade650264c87b88b"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.953935 17748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "70753283514f4381bed7e3f3b3e8bbad"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "02753e44c7ef2a42843dae76839c6e70"
server_key_iv: "0697bce9e36fa595ade650264c87b88b"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.955232 17748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:00.955988 17763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:00.956147 17748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:00.956254 17748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "70753283514f4381bed7e3f3b3e8bbad"
format_stamp: "Formatted at 2026-06-20 18:44:00 on dist-test-slave-tvl9"
server_key: "02753e44c7ef2a42843dae76839c6e70"
server_key_iv: "0697bce9e36fa595ade650264c87b88b"
server_key_version: "encryptionkey@0"
I20260620 18:44:00.956331 17748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:00.971114 17748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:00.973035 17748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:00.973254 17748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:00.973837 17748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:00.973896 17748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:00.973935 17748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:00.973958 17748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 18:44:00 dist-test-slave-tvl9 krb5kdc[17351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:00.981472 17748 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:39855
I20260620 18:44:00.981540 17878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:39855 every 8 connection(s)
I20260620 18:44:00.981907 17748 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:44:00.983692 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.974300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:55769 (local address 127.15.77.126:36545)
0620 18:44:00.974365 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:00.974367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:00.974596 (+ 229us) server_negotiation.cc:407] Connection header received
0620 18:44:00.975373 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:00.975374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:00.975388 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:00.975486 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:00.976082 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.976957 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.977994 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.978350 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.978433 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.980800 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:00.980802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:00.980802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:00.980812 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:00.982535 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.982891 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.982892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.982892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.982944 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.983112 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.983112 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.983112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.983223 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:00.983272 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:00.983560 (+ 288us) server_negotiation.cc:299] Negotiation successful
0620 18:44:00.983624 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 18:44:00.983913 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.974310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36545 (local address 127.15.77.67:55769)
0620 18:44:00.974473 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:00.974509 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:00.975223 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:00.975566 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:00.975570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:00.975820 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:00.976006 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.976010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.977067 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.977068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.977890 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.977891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.978462 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.978464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.978580 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.979132 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:00.979140 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:00.980720 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:00.982602 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.982606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.982608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.982825 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.982984 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.982984 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.982984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.983064 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.983279 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:00.983283 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:00.983448 (+ 165us) client_negotiation.cc:769] Sending connection context
0620 18:44:00.983532 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 18:44:00.983664 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260620 18:44:00.984391 17879 heartbeater.cc:347] Connected to a master server at 127.15.77.126:36545
I20260620 18:44:00.984470 17879 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:00.984666 17879 heartbeater.cc:511] Master 127.15.77.126:36545 requested a full tablet report, sending...
I20260620 18:44:00.985083 17395 ts_manager.cc:194] Registered new tserver with Master: 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855)
I20260620 18:44:00.985551 17395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:55769
I20260620 18:44:00.989895 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 17748
I20260620 18:44:00.989987 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:44:00.990183 15669 external_mini_cluster.cc:1428] Setting key 285f146eedc50068ae17845ca9b6445a
I20260620 18:44:00.991199 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:44:00.991386 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.985410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36545 (local address 127.15.77.67:40727)
0620 18:44:00.985731 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:00.985742 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:00.985796 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:00.985929 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:00.985929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:00.986121 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:00.986186 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.986188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.986799 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.986800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.987284 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:00.987285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.987697 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.987698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:00.987755 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.988119 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:00.988123 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:00.988389 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:00.990076 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.990077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.990077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.990291 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.990505 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:00.990505 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:00.990506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:00.990576 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:00.991004 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:00.991005 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:00.991060 (+ 55us) client_negotiation.cc:769] Sending connection context
0620 18:44:00.991138 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 18:44:00.991268 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":72,"threads_started":1}
I20260620 18:44:00.991398 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:00.985482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:40727 (local address 127.15.77.126:36545)
0620 18:44:00.985516 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:00.985518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:00.985800 (+ 282us) server_negotiation.cc:407] Connection header received
0620 18:44:00.985862 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:00.985863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:00.985875 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:00.985925 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:00.986259 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.986710 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.987432 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:00.987629 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:00.987690 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:00.988466 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:00.988467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:00.988467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:00.988473 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:00.989960 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.990374 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.990375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.990376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.990439 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:00.990700 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:00.990701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:00.990701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:00.990871 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:00.990942 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:00.991183 (+ 241us) server_negotiation.cc:299] Negotiation successful
0620 18:44:00.991276 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Jun 20 18:44:00 dist-test-slave-tvl9 krb5kdc[17351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 18:44:01 dist-test-slave-tvl9 krb5kdc[17351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:01.014465 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.007285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.15.77.126:36545)
0620 18:44:01.007368 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.007370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.007382 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:44:01.007478 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.007478 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.007488 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.007551 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:01.008150 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.008641 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.009411 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.009602 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.009695 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.012041 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:01.012043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:01.012043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:01.012050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:01.013449 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:01.013813 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:01.013814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:01.013814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:01.013872 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:01.014042 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:01.014042 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:01.014042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:01.014150 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:01.014225 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.014337 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.014397 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 18:44:01.016129 17395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:44:01.017745 17395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:01.024502 17901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.020581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53976 (local address 127.15.77.65:37547)
0620 18:44:01.020739 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.020743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.020752 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:44:01.020787 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.020789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.020802 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.020880 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:01.021099 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.021809 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.023223 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.024057 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.024167 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.024239 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.024345 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.024382 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260620 18:44:01.024492 17900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.020391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:37547 (local address 127.0.0.1:53976)
0620 18:44:01.020580 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.020592 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.020649 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.020968 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.020969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.020971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:01.021019 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.021021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.021902 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.021903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.023138 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.023140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.024167 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.024168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.024227 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.024246 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.024327 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.024358 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260620 18:44:01.024600 17897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.020199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:39855 (local address 127.0.0.1:34214)
0620 18:44:01.020507 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.020516 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.020596 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.020811 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.020814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.020816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:01.020885 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.020887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.022225 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.022227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.023316 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.023318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.024227 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.024228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.024302 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.024308 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.024340 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.024382 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":41,"thread_start_us":56,"threads_started":1}
I20260620 18:44:01.024493 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.020199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:37075 (local address 127.0.0.1:47226)
0620 18:44:01.020363 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.020403 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.020467 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.021036 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.021037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.021039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:01.021075 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.021076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.022298 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.022299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.023383 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.023386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.024139 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.024140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.024227 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.024246 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.024298 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.024354 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260620 18:44:01.024827 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.020287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.15.77.67:39855)
0620 18:44:01.020636 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.020641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.020651 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:01.020694 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.020696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.020713 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.020794 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:01.020983 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.022126 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.023427 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.024138 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.024537 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.024606 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.024652 (+ 46us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.024702 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":92,"threads_started":1}
I20260620 18:44:01.024901 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.020327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.15.77.66:37075)
0620 18:44:01.020778 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.020782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.020792 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:01.020829 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.020831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.020913 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.021143 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:01.021201 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.022216 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.023446 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.024018 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.024625 (+ 607us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.024702 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.024733 (+ 31us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.024774 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":75,"threads_started":1}
I20260620 18:44:01.025552 17813 tablet_service.cc:1467] Processing CreateTablet for tablet 54f86fd32e6946ed9ad842ab89ba7038 (DEFAULT_TABLE table=test-table [id=4957586f3bda4a58b2fb83dc74cd0976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:01.025586 17673 tablet_service.cc:1467] Processing CreateTablet for tablet 54f86fd32e6946ed9ad842ab89ba7038 (DEFAULT_TABLE table=test-table [id=4957586f3bda4a58b2fb83dc74cd0976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:01.025967 17673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54f86fd32e6946ed9ad842ab89ba7038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:01.025971 17813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54f86fd32e6946ed9ad842ab89ba7038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:01.025918 17533 tablet_service.cc:1467] Processing CreateTablet for tablet 54f86fd32e6946ed9ad842ab89ba7038 (DEFAULT_TABLE table=test-table [id=4957586f3bda4a58b2fb83dc74cd0976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:01.026233 17533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54f86fd32e6946ed9ad842ab89ba7038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:01.029204 17902 tablet_bootstrap.cc:492] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: Bootstrap starting.
I20260620 18:44:01.029414 17903 tablet_bootstrap.cc:492] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: Bootstrap starting.
I20260620 18:44:01.029870 17904 tablet_bootstrap.cc:492] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: Bootstrap starting.
I20260620 18:44:01.030028 17902 tablet_bootstrap.cc:654] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:01.030387 17902 log.cc:824] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:01.030447 17903 tablet_bootstrap.cc:654] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:01.030478 17904 tablet_bootstrap.cc:654] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:01.030755 17903 log.cc:824] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:01.030824 17904 log.cc:824] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:01.031247 17902 tablet_bootstrap.cc:492] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: No bootstrap required, opened a new log
I20260620 18:44:01.031338 17902 ts_tablet_manager.cc:1397] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:44:01.031548 17904 tablet_bootstrap.cc:492] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: No bootstrap required, opened a new log
I20260620 18:44:01.031556 17903 tablet_bootstrap.cc:492] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: No bootstrap required, opened a new log
I20260620 18:44:01.031621 17903 ts_tablet_manager.cc:1397] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:44:01.031621 17904 ts_tablet_manager.cc:1397] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:44:01.033011 17902 raft_consensus.cc:348] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } }
I20260620 18:44:01.033176 17902 raft_consensus.cc:374] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:01.033221 17902 raft_consensus.cc:729] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84d7b54d43948dbb4634a64af7bbd2b, State: Initialized, Role: FOLLOWER
I20260620 18:44:01.033351 17902 consensus_queue.cc:260] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } }
I20260620 18:44:01.033560 17904 raft_consensus.cc:348] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } }
I20260620 18:44:01.033565 17903 raft_consensus.cc:348] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } }
I20260620 18:44:01.033674 17739 heartbeater.cc:503] Master 127.15.77.126:36545 was elected leader, sending a full tablet report...
I20260620 18:44:01.033721 17903 raft_consensus.cc:374] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:01.033721 17904 raft_consensus.cc:374] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:01.033764 17904 raft_consensus.cc:729] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99b7f4a80a8471d885cb4652fefdc6c, State: Initialized, Role: FOLLOWER
I20260620 18:44:01.033764 17903 raft_consensus.cc:729] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70753283514f4381bed7e3f3b3e8bbad, State: Initialized, Role: FOLLOWER
I20260620 18:44:01.033879 17903 consensus_queue.cc:260] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } }
I20260620 18:44:01.033892 17904 consensus_queue.cc:260] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } }
I20260620 18:44:01.033921 17902 ts_tablet_manager.cc:1428] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 18:44:01.034145 17903 ts_tablet_manager.cc:1428] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 18:44:01.034145 17904 ts_tablet_manager.cc:1428] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 18:44:01.034262 17599 heartbeater.cc:503] Master 127.15.77.126:36545 was elected leader, sending a full tablet report...
I20260620 18:44:01.034262 17879 heartbeater.cc:503] Master 127.15.77.126:36545 was elected leader, sending a full tablet report...
W20260620 18:44:01.069945 17740 tablet.cc:2369] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 18:44:01.163151 17600 tablet.cc:2369] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 18:44:01.232520 17880 tablet.cc:2369] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:44:01.238790 17910 raft_consensus.cc:482] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:01.238914 17910 raft_consensus.cc:504] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } }
I20260620 18:44:01.239281 17910 leader_election.cc:288] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075), f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65:37547)
I20260620 18:44:01.243263 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.239644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:37251 (local address 127.15.77.66:37075)
0620 18:44:01.239692 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.239695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.239850 (+ 155us) server_negotiation.cc:407] Connection header received
0620 18:44:01.239901 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.239902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.239909 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.239967 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:01.240112 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.240793 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.242312 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.242877 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.242958 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.242966 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.243147 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.243188 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 18:44:01.243263 17901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.239638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:32845 (local address 127.15.77.65:37547)
0620 18:44:01.239696 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.239698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.239708 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:01.239746 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.239747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.239755 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.239807 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:01.239989 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.240781 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.242313 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.242877 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.242957 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.242966 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.243154 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.243192 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 18:44:01.243256 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.239514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:37547 (local address 127.15.77.67:32845)
0620 18:44:01.239581 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.239594 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.239647 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.239835 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.239837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.239839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:01.239910 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.239912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.240869 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.240870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.242211 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.242214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.242966 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.242968 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.243084 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.243086 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.243138 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.243182 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":46}
I20260620 18:44:01.243264 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.239526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:37075 (local address 127.15.77.67:37251)
0620 18:44:01.239749 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.239757 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.239836 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.239962 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.239962 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.239964 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:01.240026 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.240028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.240869 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.240870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.242211 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.242214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.242967 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.242968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.243049 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.243053 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.243115 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.243161 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":136,"thread_start_us":106,"threads_started":1}
I20260620 18:44:01.243680 17693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f86fd32e6946ed9ad842ab89ba7038" candidate_uuid: "70753283514f4381bed7e3f3b3e8bbad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" is_pre_election: true
I20260620 18:44:01.243671 17553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f86fd32e6946ed9ad842ab89ba7038" candidate_uuid: "70753283514f4381bed7e3f3b3e8bbad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" is_pre_election: true
I20260620 18:44:01.243847 17693 raft_consensus.cc:2454] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70753283514f4381bed7e3f3b3e8bbad in term 0.
I20260620 18:44:01.243857 17553 raft_consensus.cc:2454] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70753283514f4381bed7e3f3b3e8bbad in term 0.
I20260620 18:44:01.244108 17764 leader_election.cc:302] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70753283514f4381bed7e3f3b3e8bbad, f84d7b54d43948dbb4634a64af7bbd2b; no voters:
I20260620 18:44:01.244240 17910 raft_consensus.cc:2781] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:01.244292 17910 raft_consensus.cc:482] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:01.244316 17910 raft_consensus.cc:3037] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:01.245239 17910 raft_consensus.cc:504] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } }
I20260620 18:44:01.245393 17910 leader_election.cc:288] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [CANDIDATE]: Term 1 election: Requested vote from peers f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075), f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65:37547)
I20260620 18:44:01.245553 17553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f86fd32e6946ed9ad842ab89ba7038" candidate_uuid: "70753283514f4381bed7e3f3b3e8bbad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99b7f4a80a8471d885cb4652fefdc6c"
I20260620 18:44:01.245545 17693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f86fd32e6946ed9ad842ab89ba7038" candidate_uuid: "70753283514f4381bed7e3f3b3e8bbad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84d7b54d43948dbb4634a64af7bbd2b"
I20260620 18:44:01.245638 17693 raft_consensus.cc:3037] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:01.245638 17553 raft_consensus.cc:3037] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:01.246565 17693 raft_consensus.cc:2454] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70753283514f4381bed7e3f3b3e8bbad in term 1.
I20260620 18:44:01.246577 17553 raft_consensus.cc:2454] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70753283514f4381bed7e3f3b3e8bbad in term 1.
I20260620 18:44:01.246740 17765 leader_election.cc:302] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70753283514f4381bed7e3f3b3e8bbad, f99b7f4a80a8471d885cb4652fefdc6c; no voters:
I20260620 18:44:01.246858 17910 raft_consensus.cc:2781] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:01.247014 17910 raft_consensus.cc:686] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 1 LEADER]: Becoming Leader. State: Replica: 70753283514f4381bed7e3f3b3e8bbad, State: Running, Role: LEADER
I20260620 18:44:01.247129 17910 consensus_queue.cc:237] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } }
I20260620 18:44:01.247982 17396 catalog_manager.cc:5495] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad reported cstate change: term changed from 0 to 1, leader changed from <none> to 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67). New cstate: current_term: 1 leader_uuid: "70753283514f4381bed7e3f3b3e8bbad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } health_report { overall_health: UNKNOWN } } }
Jun 20 18:44:01 dist-test-slave-tvl9 krb5kdc[17351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981040, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:01.335453 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.328252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43534 (local address 127.15.77.126:36545)
0620 18:44:01.328342 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.328345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.328499 (+ 154us) server_negotiation.cc:407] Connection header received
0620 18:44:01.328615 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.328625 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.328634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.328693 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:01.329201 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.329775 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.330496 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.330677 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.330763 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.332885 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:01.332887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:01.332887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:01.332894 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:01.334130 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:01.334542 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:01.334544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:01.334544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:01.334599 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:01.334800 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:01.334801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:01.334802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:01.334914 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:01.334996 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.335166 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.335376 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":55}
I20260620 18:44:01.335455 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.328205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36545 (local address 127.0.0.1:43534)
0620 18:44:01.328393 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.328404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.328522 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.328712 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.328713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.329051 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:01.329100 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.329102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.329862 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.329863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.330388 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.330389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.330768 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.330770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.330832 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.331431 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:01.331436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:01.332785 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:01.334230 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:01.334238 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:01.334241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:01.334445 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:01.334665 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:01.334665 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:01.334666 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:01.334725 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:01.334997 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:01.335006 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:01.335053 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.335139 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.335293 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260620 18:44:01.337616 17396 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:43534:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 18:44:01.337858 17396 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:01.339730 17813 tablet_service.cc:1467] Processing CreateTablet for tablet bb43be79f6a1429680f7066cba83fc36 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80221702bfc543308b6a074f8568d229]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 18:44:01.339720 17533 tablet_service.cc:1467] Processing CreateTablet for tablet bb43be79f6a1429680f7066cba83fc36 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80221702bfc543308b6a074f8568d229]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 18:44:01.339748 17673 tablet_service.cc:1467] Processing CreateTablet for tablet bb43be79f6a1429680f7066cba83fc36 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80221702bfc543308b6a074f8568d229]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 18:44:01.339903 17813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb43be79f6a1429680f7066cba83fc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:01.339903 17533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb43be79f6a1429680f7066cba83fc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:01.339918 17673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb43be79f6a1429680f7066cba83fc36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:01.341174 17902 tablet_bootstrap.cc:492] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: Bootstrap starting.
I20260620 18:44:01.341439 17904 tablet_bootstrap.cc:492] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: Bootstrap starting.
I20260620 18:44:01.341619 17902 tablet_bootstrap.cc:654] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:01.341878 17904 tablet_bootstrap.cc:654] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:01.341907 17903 tablet_bootstrap.cc:492] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: Bootstrap starting.
I20260620 18:44:01.342290 17902 tablet_bootstrap.cc:492] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: No bootstrap required, opened a new log
I20260620 18:44:01.342303 17903 tablet_bootstrap.cc:654] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:01.342337 17902 ts_tablet_manager.cc:1397] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:01.342450 17904 tablet_bootstrap.cc:492] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: No bootstrap required, opened a new log
I20260620 18:44:01.342496 17904 ts_tablet_manager.cc:1397] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:01.342520 17902 raft_consensus.cc:348] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.342609 17902 raft_consensus.cc:374] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:01.342635 17902 raft_consensus.cc:729] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84d7b54d43948dbb4634a64af7bbd2b, State: Initialized, Role: FOLLOWER
I20260620 18:44:01.342604 17904 raft_consensus.cc:348] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.342687 17904 raft_consensus.cc:374] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:01.342711 17904 raft_consensus.cc:729] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99b7f4a80a8471d885cb4652fefdc6c, State: Initialized, Role: FOLLOWER
I20260620 18:44:01.342685 17902 consensus_queue.cc:260] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.342814 17902 ts_tablet_manager.cc:1428] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:01.342782 17904 consensus_queue.cc:260] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.342913 17904 ts_tablet_manager.cc:1428] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:01.343132 17903 tablet_bootstrap.cc:492] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: No bootstrap required, opened a new log
I20260620 18:44:01.343123 17908 tablet_replica.cc:439] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } }
I20260620 18:44:01.343189 17903 ts_tablet_manager.cc:1397] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:01.343209 17908 tablet_replica.cc:442] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:01.343351 17903 raft_consensus.cc:348] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.343441 17903 raft_consensus.cc:374] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:01.343463 17903 raft_consensus.cc:729] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70753283514f4381bed7e3f3b3e8bbad, State: Initialized, Role: FOLLOWER
I20260620 18:44:01.343403 17909 tablet_replica.cc:439] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } }
I20260620 18:44:01.343495 17909 tablet_replica.cc:442] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:01.343508 17903 consensus_queue.cc:260] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.343636 17903 ts_tablet_manager.cc:1428] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:01.343756 17910 tablet_replica.cc:439] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } }
I20260620 18:44:01.343834 17910 tablet_replica.cc:442] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:01.480381 17908 raft_consensus.cc:482] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:01.480499 17908 raft_consensus.cc:504] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.480899 17908 leader_election.cc:288] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65:37547), 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855)
I20260620 18:44:01.484088 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.481271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:39855 (local address 127.15.77.66:39931)
0620 18:44:01.481390 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.481400 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.481458 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.481609 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.481610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.481612 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:01.481690 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.481692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.482345 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.482346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.483186 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.483188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.483799 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.483800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.483891 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.483894 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.483956 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.483997 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":54,"threads_started":1}
I20260620 18:44:01.484078 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.481352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:39931 (local address 127.15.77.67:39855)
0620 18:44:01.481403 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.481406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.481441 (+ 35us) server_negotiation.cc:407] Connection header received
0620 18:44:01.481513 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.481514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.481524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.481603 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:01.481769 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.482254 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.483290 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.483696 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.483780 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.483800 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.483975 (+ 175us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.484011 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:44:01.484380 17901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.481222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:52491 (local address 127.15.77.65:37547)
0620 18:44:01.481277 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.481280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.481342 (+ 62us) server_negotiation.cc:407] Connection header received
0620 18:44:01.481476 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.481477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.481494 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.481564 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:01.481705 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.482200 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.483233 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.483670 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.483757 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.483783 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.484268 (+ 485us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.484311 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 18:44:01.484414 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.481087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:37547 (local address 127.15.77.66:52491)
0620 18:44:01.481246 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.481260 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.481360 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.481567 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.481568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.481571 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:01.481627 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.481629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.482289 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.482290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.483132 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.483135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.484060 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.484061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.484124 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.484145 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.484208 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.484261 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":63,"threads_started":1}
I20260620 18:44:01.484472 17833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb43be79f6a1429680f7066cba83fc36" candidate_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70753283514f4381bed7e3f3b3e8bbad" is_pre_election: true
I20260620 18:44:01.484612 17833 raft_consensus.cc:2454] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f84d7b54d43948dbb4634a64af7bbd2b in term 0.
I20260620 18:44:01.484648 17553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb43be79f6a1429680f7066cba83fc36" candidate_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" is_pre_election: true
I20260620 18:44:01.484768 17553 raft_consensus.cc:2454] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f84d7b54d43948dbb4634a64af7bbd2b in term 0.
I20260620 18:44:01.484964 17625 leader_election.cc:302] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70753283514f4381bed7e3f3b3e8bbad, f84d7b54d43948dbb4634a64af7bbd2b; no voters:
I20260620 18:44:01.485110 17908 raft_consensus.cc:2781] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:01.485152 17908 raft_consensus.cc:482] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:01.485183 17908 raft_consensus.cc:3037] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:01.485877 17908 raft_consensus.cc:504] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.486014 17908 leader_election.cc:288] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [CANDIDATE]: Term 1 election: Requested vote from peers f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65:37547), 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855)
I20260620 18:44:01.486181 17833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb43be79f6a1429680f7066cba83fc36" candidate_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70753283514f4381bed7e3f3b3e8bbad"
I20260620 18:44:01.486200 17553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb43be79f6a1429680f7066cba83fc36" candidate_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99b7f4a80a8471d885cb4652fefdc6c"
I20260620 18:44:01.486272 17553 raft_consensus.cc:3037] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:01.486274 17833 raft_consensus.cc:3037] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:01.487093 17553 raft_consensus.cc:2454] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f84d7b54d43948dbb4634a64af7bbd2b in term 1.
I20260620 18:44:01.487093 17833 raft_consensus.cc:2454] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f84d7b54d43948dbb4634a64af7bbd2b in term 1.
I20260620 18:44:01.487362 17625 leader_election.cc:302] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f84d7b54d43948dbb4634a64af7bbd2b, f99b7f4a80a8471d885cb4652fefdc6c; no voters:
I20260620 18:44:01.487504 17908 raft_consensus.cc:2781] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:01.487685 17908 raft_consensus.cc:686] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [term 1 LEADER]: Becoming Leader. State: Replica: f84d7b54d43948dbb4634a64af7bbd2b, State: Running, Role: LEADER
I20260620 18:44:01.487800 17908 consensus_queue.cc:237] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.488265 17925 tablet_replica.cc:439] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: TxnStatusTablet state changed. Reason: New leader f84d7b54d43948dbb4634a64af7bbd2b. Latest consensus state: current_term: 1 leader_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } }
I20260620 18:44:01.488358 17925 tablet_replica.cc:442] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: This TxnStatusTablet replica's current role is: LEADER
I20260620 18:44:01.488535 17927 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 18:44:01.488644 17927 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 18:44:01.488647 17396 catalog_manager.cc:5495] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66). New cstate: current_term: 1 leader_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } health_report { overall_health: UNKNOWN } } }
I20260620 18:44:01.529014 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.525930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.15.77.66:37075)
0620 18:44:01.525970 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.525972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.526009 (+ 37us) server_negotiation.cc:407] Connection header received
0620 18:44:01.526115 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.526116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.526153 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.526231 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:01.526395 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.526903 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.528233 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.528416 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.528490 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.528687 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.528789 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.528852 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.528922 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.528940 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 18:44:01.529063 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.525878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:37075 (local address 127.0.0.1:34746)
0620 18:44:01.525966 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.525977 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.526036 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.526248 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.526249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.526250 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:44:01.526305 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.526307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.526980 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.526981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.528123 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.528127 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.528502 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.528503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.528573 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.528592 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.528855 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.528863 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.528920 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.528969 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260620 18:44:01.531044 17833 raft_consensus.cc:1261] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Refusing update from remote peer f84d7b54d43948dbb4634a64af7bbd2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:01.531040 17553 raft_consensus.cc:1261] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Refusing update from remote peer f84d7b54d43948dbb4634a64af7bbd2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:01.531373 17908 consensus_queue.cc:1035] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:01.531525 17925 consensus_queue.cc:1035] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:01.532562 17910 tablet_replica.cc:439] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: TxnStatusTablet state changed. Reason: New leader f84d7b54d43948dbb4634a64af7bbd2b. Latest consensus state: current_term: 1 leader_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } }
I20260620 18:44:01.532655 17910 tablet_replica.cc:442] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:01.532816 17909 tablet_replica.cc:439] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: TxnStatusTablet state changed. Reason: New leader f84d7b54d43948dbb4634a64af7bbd2b. Latest consensus state: current_term: 1 leader_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } }
I20260620 18:44:01.532900 17909 tablet_replica.cc:442] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:01.533579 17925 tablet_replica.cc:439] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } }
I20260620 18:44:01.533663 17925 tablet_replica.cc:442] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: This TxnStatusTablet replica's current role is: LEADER
I20260620 18:44:01.533677 17909 tablet_replica.cc:439] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } }
I20260620 18:44:01.533751 17909 tablet_replica.cc:442] T bb43be79f6a1429680f7066cba83fc36 P f99b7f4a80a8471d885cb4652fefdc6c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:01.533775 17930 mvcc.cc:205] Tried to move back new op lower bound from 7298994346108411904 to 7298994345934663680. Current Snapshot: MvccSnapshot[applied={T|T < 7298994346108411904}]
I20260620 18:44:01.534090 17932 mvcc.cc:205] Tried to move back new op lower bound from 7298994346108411904 to 7298994345934663680. Current Snapshot: MvccSnapshot[applied={T|T < 7298994346108411904}]
I20260620 18:44:01.538580 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.535812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.15.77.67:39855)
0620 18:44:01.535856 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.535858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.535994 (+ 136us) server_negotiation.cc:407] Connection header received
0620 18:44:01.536077 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.536077 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.536093 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.536149 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:01.536265 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.536720 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.537843 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.538049 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.538106 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.538274 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.538367 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.538423 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.538486 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.538501 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:44:01.543988 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.541207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:60831 (local address 127.15.77.66:37075)
0620 18:44:01.541275 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.541277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.541349 (+ 72us) server_negotiation.cc:407] Connection header received
0620 18:44:01.541436 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.541436 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.541445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.541486 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:01.541608 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.542048 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.543151 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.543425 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.543503 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.543747 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.543792 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.543836 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.543899 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.543919 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 18:44:01.544054 17941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.541112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:37075 (local address 127.15.77.67:60831)
0620 18:44:01.541291 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.541304 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.541375 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.541497 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.541498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.541499 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:44:01.541553 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.541555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.542122 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.542123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.543049 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.543052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.543517 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.543518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.543594 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.543612 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.543851 (+ 239us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.543853 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.543897 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.543943 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":102,"threads_started":1}
I20260620 18:44:01.544032 17925 tablet_replica.cc:439] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } }
I20260620 18:44:01.544135 17925 tablet_replica.cc:442] T bb43be79f6a1429680f7066cba83fc36 P f84d7b54d43948dbb4634a64af7bbd2b: This TxnStatusTablet replica's current role is: LEADER
I20260620 18:44:01.544786 17910 tablet_replica.cc:439] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } } }
I20260620 18:44:01.544878 17910 tablet_replica.cc:442] T bb43be79f6a1429680f7066cba83fc36 P 70753283514f4381bed7e3f3b3e8bbad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:01.547195 17693 raft_consensus.cc:1261] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Refusing update from remote peer 70753283514f4381bed7e3f3b3e8bbad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:01.547216 17553 raft_consensus.cc:1261] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Refusing update from remote peer 70753283514f4381bed7e3f3b3e8bbad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:01.547528 17912 consensus_queue.cc:1035] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [LEADER]: Connected to new peer: Peer: permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:01.547762 17912 consensus_queue.cc:1035] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:01.548942 17931 mvcc.cc:205] Tried to move back new op lower bound from 7298994346175168512 to 7298994344948838400. Current Snapshot: MvccSnapshot[applied={T|T < 7298994346175168512}]
I20260620 18:44:01.556983 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.554251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:39855 (local address 127.15.77.66:60137)
0620 18:44:01.554408 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.554421 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.554474 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.554596 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.554597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.554598 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:44:01.554646 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.554647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.555250 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.555251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.556097 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.556100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.556592 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.556593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.556639 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.556653 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.556848 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.556850 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.556885 (+ 35us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.556922 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260620 18:44:01.556988 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.554448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:60137 (local address 127.15.77.67:39855)
0620 18:44:01.554496 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.554498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.554506 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:44:01.554528 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.554528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.554538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.554593 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:01.554694 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.555159 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.556189 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.556489 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.556577 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.556743 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.556780 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.556842 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.556891 (+ 49us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.556914 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:44:01.566239 17395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:01.566363 17395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:01.567230 17395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454: Sending DeleteTablet for 3 replicas of tablet 54f86fd32e6946ed9ad842ab89ba7038
I20260620 18:44:01.567617 17533 tablet_service.cc:1514] Processing DeleteTablet for tablet 54f86fd32e6946ed9ad842ab89ba7038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:01 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:53976
I20260620 18:44:01.567648 17813 tablet_service.cc:1514] Processing DeleteTablet for tablet 54f86fd32e6946ed9ad842ab89ba7038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:01 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:34214
I20260620 18:44:01.567863 17673 tablet_service.cc:1514] Processing DeleteTablet for tablet 54f86fd32e6946ed9ad842ab89ba7038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:01 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:47226
I20260620 18:44:01.568112 17951 tablet_replica.cc:330] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: stopping tablet replica
I20260620 18:44:01.568225 17951 raft_consensus.cc:2229] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:01.568285 17951 raft_consensus.cc:2258] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:01.568512 17395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:44:01.568675 17395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:01.568682 17951 ts_tablet_manager.cc:1905] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:01.570313 17813 tablet_service.cc:1467] Processing CreateTablet for tablet fdbe20b9e7eb4608b2285beab173ee9b (DEFAULT_TABLE table=test-table [id=afa05d79ed6e4160bb12436fc3e106e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:01.570441 17813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbe20b9e7eb4608b2285beab173ee9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:01.570638 17951 ts_tablet_manager.cc:1918] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:01.570701 17951 log.cc:1196] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/54f86fd32e6946ed9ad842ab89ba7038
I20260620 18:44:01.570972 17951 ts_tablet_manager.cc:1939] T 54f86fd32e6946ed9ad842ab89ba7038 P f84d7b54d43948dbb4634a64af7bbd2b: Deleting consensus metadata
I20260620 18:44:01.571410 17380 catalog_manager.cc:4841] TS f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075): tablet 54f86fd32e6946ed9ad842ab89ba7038 (table test-table [id=4957586f3bda4a58b2fb83dc74cd0976]) successfully deleted
I20260620 18:44:01.571468 17533 tablet_service.cc:1467] Processing CreateTablet for tablet fdbe20b9e7eb4608b2285beab173ee9b (DEFAULT_TABLE table=test-table [id=afa05d79ed6e4160bb12436fc3e106e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:01.571592 17533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbe20b9e7eb4608b2285beab173ee9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:01.572907 17904 tablet_bootstrap.cc:492] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c: Bootstrap starting.
I20260620 18:44:01.573601 17904 tablet_bootstrap.cc:654] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:01.574503 17673 tablet_service.cc:1467] Processing CreateTablet for tablet fdbe20b9e7eb4608b2285beab173ee9b (DEFAULT_TABLE table=test-table [id=afa05d79ed6e4160bb12436fc3e106e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:01.574625 17673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbe20b9e7eb4608b2285beab173ee9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:01.575910 17902 tablet_bootstrap.cc:492] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b: Bootstrap starting.
I20260620 18:44:01.576289 17902 tablet_bootstrap.cc:654] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:01.577178 17902 tablet_bootstrap.cc:492] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b: No bootstrap required, opened a new log
I20260620 18:44:01.577227 17902 ts_tablet_manager.cc:1397] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:01.577376 17902 raft_consensus.cc:348] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.577445 17902 raft_consensus.cc:374] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:01.577464 17902 raft_consensus.cc:729] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84d7b54d43948dbb4634a64af7bbd2b, State: Initialized, Role: FOLLOWER
I20260620 18:44:01.577507 17902 consensus_queue.cc:260] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.577594 17902 ts_tablet_manager.cc:1428] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:01.578066 17904 tablet_bootstrap.cc:492] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c: No bootstrap required, opened a new log
I20260620 18:44:01.578121 17904 ts_tablet_manager.cc:1397] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260620 18:44:01.578263 17904 raft_consensus.cc:348] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.578326 17904 raft_consensus.cc:374] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:01.578346 17904 raft_consensus.cc:729] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99b7f4a80a8471d885cb4652fefdc6c, State: Initialized, Role: FOLLOWER
I20260620 18:44:01.578388 17904 consensus_queue.cc:260] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.578476 17904 ts_tablet_manager.cc:1428] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:01.579440 17950 tablet_replica.cc:330] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: stopping tablet replica
I20260620 18:44:01.579602 17950 raft_consensus.cc:2229] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:44:01.579732 17950 raft_consensus.cc:2258] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:01.580174 17903 tablet_bootstrap.cc:492] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad: Bootstrap starting.
I20260620 18:44:01.580621 17903 tablet_bootstrap.cc:654] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:01.580937 17950 ts_tablet_manager.cc:1905] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:01.582957 17950 ts_tablet_manager.cc:1918] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:01.583086 17950 log.cc:1196] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/54f86fd32e6946ed9ad842ab89ba7038
I20260620 18:44:01.583433 17950 ts_tablet_manager.cc:1939] T 54f86fd32e6946ed9ad842ab89ba7038 P 70753283514f4381bed7e3f3b3e8bbad: Deleting consensus metadata
I20260620 18:44:01.583904 17381 catalog_manager.cc:4841] TS 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855): tablet 54f86fd32e6946ed9ad842ab89ba7038 (table test-table [id=4957586f3bda4a58b2fb83dc74cd0976]) successfully deleted
I20260620 18:44:01.584434 17949 tablet_replica.cc:330] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: stopping tablet replica
I20260620 18:44:01.584625 17949 raft_consensus.cc:2229] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:01.584728 17949 raft_consensus.cc:2258] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:01.584860 17903 tablet_bootstrap.cc:492] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad: No bootstrap required, opened a new log
I20260620 18:44:01.584913 17903 ts_tablet_manager.cc:1397] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260620 18:44:01.585065 17903 raft_consensus.cc:348] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.585143 17903 raft_consensus.cc:374] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:01.585173 17903 raft_consensus.cc:729] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70753283514f4381bed7e3f3b3e8bbad, State: Initialized, Role: FOLLOWER
I20260620 18:44:01.585213 17903 consensus_queue.cc:260] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.585312 17903 ts_tablet_manager.cc:1428] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:01.585431 17949 ts_tablet_manager.cc:1905] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:01.587224 17949 ts_tablet_manager.cc:1918] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:01.587374 17949 log.cc:1196] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/54f86fd32e6946ed9ad842ab89ba7038
I20260620 18:44:01.587708 17949 ts_tablet_manager.cc:1939] T 54f86fd32e6946ed9ad842ab89ba7038 P f99b7f4a80a8471d885cb4652fefdc6c: Deleting consensus metadata
I20260620 18:44:01.588177 17381 catalog_manager.cc:4841] TS f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65:37547): tablet 54f86fd32e6946ed9ad842ab89ba7038 (table test-table [id=4957586f3bda4a58b2fb83dc74cd0976]) successfully deleted
I20260620 18:44:01.877085 17909 raft_consensus.cc:482] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:01.877254 17909 raft_consensus.cc:504] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.877633 17909 leader_election.cc:288] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075), 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855)
I20260620 18:44:01.880730 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.877915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:59187 (local address 127.15.77.67:39855)
0620 18:44:01.877969 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.877973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.878079 (+ 106us) server_negotiation.cc:407] Connection header received
0620 18:44:01.878156 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.878158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.878169 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.878238 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:01.878418 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.878926 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.880003 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.880364 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.880461 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.880471 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.880622 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.880656 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 18:44:01.880734 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.877945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:45077 (local address 127.15.77.66:37075)
0620 18:44:01.877999 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.878001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.878102 (+ 101us) server_negotiation.cc:407] Connection header received
0620 18:44:01.878183 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.878184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.878193 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.878240 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:01.878396 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.878880 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.880012 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.880369 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.880454 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.880465 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.880625 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.880660 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 18:44:01.880753 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.877830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:39855 (local address 127.15.77.65:59187)
0620 18:44:01.877991 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.878002 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.878093 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.878253 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.878254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.878256 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:01.878346 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.878348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.879040 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.879041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.879876 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.879879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.880455 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.880456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.880519 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.880523 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.880608 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.880652 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":66,"thread_start_us":77,"threads_started":1}
I20260620 18:44:01.880756 17956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.877852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:37075 (local address 127.15.77.65:45077)
0620 18:44:01.878015 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.878026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.878115 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.878242 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.878243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.878245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:01.878325 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.878327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.878972 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.878973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.879912 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.879915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.880465 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.880466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.880543 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.880546 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.880620 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.880660 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":66,"threads_started":1}
I20260620 18:44:01.881008 17833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbe20b9e7eb4608b2285beab173ee9b" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70753283514f4381bed7e3f3b3e8bbad" is_pre_election: true
I20260620 18:44:01.881125 17833 raft_consensus.cc:2454] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 0.
I20260620 18:44:01.881315 17485 leader_election.cc:302] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70753283514f4381bed7e3f3b3e8bbad, f99b7f4a80a8471d885cb4652fefdc6c; no voters:
I20260620 18:44:01.881434 17693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbe20b9e7eb4608b2285beab173ee9b" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" is_pre_election: true
I20260620 18:44:01.881532 17693 raft_consensus.cc:2454] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 0.
I20260620 18:44:01.881626 17909 raft_consensus.cc:2781] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:01.881678 17909 raft_consensus.cc:482] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:01.881701 17909 raft_consensus.cc:3037] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:01.882402 17909 raft_consensus.cc:504] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.882556 17909 leader_election.cc:288] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 election: Requested vote from peers f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075), 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855)
I20260620 18:44:01.882705 17833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbe20b9e7eb4608b2285beab173ee9b" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70753283514f4381bed7e3f3b3e8bbad"
I20260620 18:44:01.882791 17833 raft_consensus.cc:3037] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:01.882758 17693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbe20b9e7eb4608b2285beab173ee9b" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84d7b54d43948dbb4634a64af7bbd2b"
I20260620 18:44:01.882880 17693 raft_consensus.cc:3037] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:01.883515 17833 raft_consensus.cc:2454] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 1.
I20260620 18:44:01.883690 17693 raft_consensus.cc:2454] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 1.
I20260620 18:44:01.883693 17485 leader_election.cc:302] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70753283514f4381bed7e3f3b3e8bbad, f99b7f4a80a8471d885cb4652fefdc6c; no voters:
I20260620 18:44:01.883810 17909 raft_consensus.cc:2781] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:01.883987 17909 raft_consensus.cc:686] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 1 LEADER]: Becoming Leader. State: Replica: f99b7f4a80a8471d885cb4652fefdc6c, State: Running, Role: LEADER
I20260620 18:44:01.884063 17909 consensus_queue.cc:237] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:01.884842 17395 catalog_manager.cc:5495] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c reported cstate change: term changed from 0 to 1, leader changed from <none> to f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65). New cstate: current_term: 1 leader_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } health_report { overall_health: UNKNOWN } } }
I20260620 18:44:01.940797 17901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.938448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48558 (local address 127.15.77.65:37547)
0620 18:44:01.938517 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.938519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.938541 (+ 22us) server_negotiation.cc:407] Connection header received
0620 18:44:01.938596 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.938597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.938619 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.938676 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:01.938786 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.939259 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.940049 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.940238 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.940297 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.940435 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.940559 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.940620 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.940656 (+ 36us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.940675 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 18:44:01.946857 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.943571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:37753 (local address 127.15.77.66:37075)
0620 18:44:01.943647 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.943650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.943789 (+ 139us) server_negotiation.cc:407] Connection header received
0620 18:44:01.943867 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.943868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.943880 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.943949 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:01.944372 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.945136 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.946062 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.946265 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.946354 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.946580 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.946625 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.946691 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.946746 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.946770 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 18:44:01.946892 17962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.943441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:37075 (local address 127.15.77.65:37753)
0620 18:44:01.943702 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.943713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.943808 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.943964 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.943965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.943967 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:44:01.944288 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.944290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.945266 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.945267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.945914 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.945916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.946374 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.946375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.946463 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.946484 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.946692 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.946695 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.946743 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.946789 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":92,"threads_started":1}
I20260620 18:44:01.949450 17693 raft_consensus.cc:1261] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Refusing update from remote peer f99b7f4a80a8471d885cb4652fefdc6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:01.949450 17833 raft_consensus.cc:1261] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Refusing update from remote peer f99b7f4a80a8471d885cb4652fefdc6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:01.949764 17957 consensus_queue.cc:1035] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:01.949909 17909 consensus_queue.cc:1035] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:01.957150 17901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.954688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:52273 (local address 127.15.77.65:37547)
0620 18:44:01.954820 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:01.954822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:01.954829 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:44:01.954852 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:01.954853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:01.954863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:01.954914 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:01.955020 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.955686 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.956422 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.956651 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.956734 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.956900 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.956935 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.956986 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:01.957055 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 18:44:01.957074 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260620 18:44:01.957186 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:01.954600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:37547 (local address 127.15.77.66:52273)
0620 18:44:01.954695 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:01.954707 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:01.954747 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:01.954919 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:01.954920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:01.954921 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:44:01.954969 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.954970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.955756 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.955757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.956329 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:01.956331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:01.956746 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:01.956747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:01.956803 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:01.956813 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:44:01.956998 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:44:01.956999 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 18:44:01.957050 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 18:44:01.957108 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
W20260620 18:44:01.957535 17932 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 18:44:03.478875 17395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:03.478976 17395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:03.479871 17395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454: Sending DeleteTablet for 3 replicas of tablet fdbe20b9e7eb4608b2285beab173ee9b
I20260620 18:44:03.480116 17533 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbe20b9e7eb4608b2285beab173ee9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:03 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:53976
I20260620 18:44:03.480151 17672 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbe20b9e7eb4608b2285beab173ee9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:03 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:47226
I20260620 18:44:03.480161 17813 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbe20b9e7eb4608b2285beab173ee9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:03 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:34214
I20260620 18:44:03.480316 17991 tablet_replica.cc:330] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b: stopping tablet replica
I20260620 18:44:03.480340 17990 tablet_replica.cc:330] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c: stopping tablet replica
I20260620 18:44:03.480365 17991 raft_consensus.cc:2229] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:03.480402 17991 raft_consensus.cc:2258] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:03.480403 17990 raft_consensus.cc:2229] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:44:03.480497 17990 raft_consensus.cc:2258] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:03.480661 17991 ts_tablet_manager.cc:1905] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:03.480757 17990 ts_tablet_manager.cc:1905] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:03.482231 17990 ts_tablet_manager.cc:1918] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 18:44:03.482288 17990 log.cc:1196] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/fdbe20b9e7eb4608b2285beab173ee9b
I20260620 18:44:03.482550 17990 ts_tablet_manager.cc:1939] T fdbe20b9e7eb4608b2285beab173ee9b P f99b7f4a80a8471d885cb4652fefdc6c: Deleting consensus metadata
I20260620 18:44:03.482576 17991 ts_tablet_manager.cc:1918] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 18:44:03.482636 17991 log.cc:1196] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/fdbe20b9e7eb4608b2285beab173ee9b
I20260620 18:44:03.482862 17991 ts_tablet_manager.cc:1939] T fdbe20b9e7eb4608b2285beab173ee9b P f84d7b54d43948dbb4634a64af7bbd2b: Deleting consensus metadata
I20260620 18:44:03.482863 17381 catalog_manager.cc:4841] TS f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65:37547): tablet fdbe20b9e7eb4608b2285beab173ee9b (table test-table [id=afa05d79ed6e4160bb12436fc3e106e6]) successfully deleted
I20260620 18:44:03.483163 17992 tablet_replica.cc:330] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad: stopping tablet replica
I20260620 18:44:03.483180 17380 catalog_manager.cc:4841] TS f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075): tablet fdbe20b9e7eb4608b2285beab173ee9b (table test-table [id=afa05d79ed6e4160bb12436fc3e106e6]) successfully deleted
I20260620 18:44:03.483256 17992 raft_consensus.cc:2229] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:03.483336 17992 raft_consensus.cc:2258] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:03.483577 17992 ts_tablet_manager.cc:1905] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:03.485481 17992 ts_tablet_manager.cc:1918] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 18:44:03.485549 17992 log.cc:1196] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/fdbe20b9e7eb4608b2285beab173ee9b
I20260620 18:44:03.485770 17992 ts_tablet_manager.cc:1939] T fdbe20b9e7eb4608b2285beab173ee9b P 70753283514f4381bed7e3f3b3e8bbad: Deleting consensus metadata
I20260620 18:44:03.486296 17381 catalog_manager.cc:4841] TS 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855): tablet fdbe20b9e7eb4608b2285beab173ee9b (table test-table [id=afa05d79ed6e4160bb12436fc3e106e6]) successfully deleted
Jun 20 18:44:03 dist-test-slave-tvl9 krb5kdc[17351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981043, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 18:44:03 dist-test-slave-tvl9 krb5kdc[17351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981043, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:03.505686 18000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:03.497426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.15.77.126:36545)
0620 18:44:03.497580 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:03.497583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:03.497595 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:44:03.497675 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:03.497676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:03.497687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:03.497775 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:03.498464 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:03.499368 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:03.500290 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:03.500517 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:03.500597 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:03.502891 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:03.502892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:03.502892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:03.502900 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:03.504444 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:03.504847 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:03.504849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:03.504849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:03.504921 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:03.505128 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:03.505129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:03.505129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:03.505256 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:03.505370 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:03.505478 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 18:44:03.505567 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":96,"threads_started":1}
I20260620 18:44:03.507179 17395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:44:03.507382 17395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:03.508952 17813 tablet_service.cc:1467] Processing CreateTablet for tablet c82b40f11e76466f9c5e8ae4c0877bcf (DEFAULT_TABLE table=test-table [id=8d4595b9c4c545f2b7f3d71ba4a430b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:03.508971 17533 tablet_service.cc:1467] Processing CreateTablet for tablet c82b40f11e76466f9c5e8ae4c0877bcf (DEFAULT_TABLE table=test-table [id=8d4595b9c4c545f2b7f3d71ba4a430b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:03.508970 17672 tablet_service.cc:1467] Processing CreateTablet for tablet c82b40f11e76466f9c5e8ae4c0877bcf (DEFAULT_TABLE table=test-table [id=8d4595b9c4c545f2b7f3d71ba4a430b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:03.509091 17813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c82b40f11e76466f9c5e8ae4c0877bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:03.509107 17672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c82b40f11e76466f9c5e8ae4c0877bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:03.509122 17533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c82b40f11e76466f9c5e8ae4c0877bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:03.510098 18001 tablet_bootstrap.cc:492] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad: Bootstrap starting.
I20260620 18:44:03.510542 18002 tablet_bootstrap.cc:492] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c: Bootstrap starting.
I20260620 18:44:03.510550 18001 tablet_bootstrap.cc:654] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:03.510942 18003 tablet_bootstrap.cc:492] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b: Bootstrap starting.
I20260620 18:44:03.510959 18002 tablet_bootstrap.cc:654] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:03.511369 18003 tablet_bootstrap.cc:654] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:03.511634 18002 tablet_bootstrap.cc:492] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c: No bootstrap required, opened a new log
I20260620 18:44:03.511691 18002 ts_tablet_manager.cc:1397] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:03.511801 18001 tablet_bootstrap.cc:492] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad: No bootstrap required, opened a new log
I20260620 18:44:03.511857 18001 ts_tablet_manager.cc:1397] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:44:03.511811 18002 raft_consensus.cc:348] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.511909 18002 raft_consensus.cc:374] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:03.511938 18002 raft_consensus.cc:729] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99b7f4a80a8471d885cb4652fefdc6c, State: Initialized, Role: FOLLOWER
I20260620 18:44:03.511991 18002 consensus_queue.cc:260] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.511978 18001 raft_consensus.cc:348] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.512043 18001 raft_consensus.cc:374] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:03.512082 18001 raft_consensus.cc:729] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70753283514f4381bed7e3f3b3e8bbad, State: Initialized, Role: FOLLOWER
I20260620 18:44:03.512104 18002 ts_tablet_manager.cc:1428] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:03.512138 18001 consensus_queue.cc:260] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.512315 18001 ts_tablet_manager.cc:1428] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:03.512514 18003 tablet_bootstrap.cc:492] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b: No bootstrap required, opened a new log
I20260620 18:44:03.512566 18003 ts_tablet_manager.cc:1397] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:44:03.512679 18003 raft_consensus.cc:348] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.512756 18003 raft_consensus.cc:374] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:03.512780 18003 raft_consensus.cc:729] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84d7b54d43948dbb4634a64af7bbd2b, State: Initialized, Role: FOLLOWER
I20260620 18:44:03.512826 18003 consensus_queue.cc:260] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.512941 18003 ts_tablet_manager.cc:1428] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:03.526265 17984 raft_consensus.cc:482] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:03.526399 17984 raft_consensus.cc:504] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.526571 17984 leader_election.cc:288] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075), 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855)
I20260620 18:44:03.526750 17693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c82b40f11e76466f9c5e8ae4c0877bcf" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" is_pre_election: true
I20260620 18:44:03.526854 17693 raft_consensus.cc:2454] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 0.
I20260620 18:44:03.526801 17833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c82b40f11e76466f9c5e8ae4c0877bcf" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70753283514f4381bed7e3f3b3e8bbad" is_pre_election: true
I20260620 18:44:03.526940 17833 raft_consensus.cc:2454] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 0.
I20260620 18:44:03.527037 17484 leader_election.cc:302] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f84d7b54d43948dbb4634a64af7bbd2b, f99b7f4a80a8471d885cb4652fefdc6c; no voters:
I20260620 18:44:03.527123 17984 raft_consensus.cc:2781] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:03.527163 17984 raft_consensus.cc:482] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:03.527182 17984 raft_consensus.cc:3037] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:03.527938 17984 raft_consensus.cc:504] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.528087 17984 leader_election.cc:288] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 election: Requested vote from peers f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075), 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855)
I20260620 18:44:03.528205 17693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c82b40f11e76466f9c5e8ae4c0877bcf" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84d7b54d43948dbb4634a64af7bbd2b"
I20260620 18:44:03.528219 17833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c82b40f11e76466f9c5e8ae4c0877bcf" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70753283514f4381bed7e3f3b3e8bbad"
I20260620 18:44:03.528296 17693 raft_consensus.cc:3037] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:03.528297 17833 raft_consensus.cc:3037] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:03.528919 17693 raft_consensus.cc:2454] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 1.
I20260620 18:44:03.528923 17833 raft_consensus.cc:2454] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 1.
I20260620 18:44:03.529134 17484 leader_election.cc:302] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f84d7b54d43948dbb4634a64af7bbd2b, f99b7f4a80a8471d885cb4652fefdc6c; no voters:
I20260620 18:44:03.529246 17984 raft_consensus.cc:2781] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:03.529309 17984 raft_consensus.cc:686] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 1 LEADER]: Becoming Leader. State: Replica: f99b7f4a80a8471d885cb4652fefdc6c, State: Running, Role: LEADER
I20260620 18:44:03.529361 17984 consensus_queue.cc:237] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.530005 17396 catalog_manager.cc:5495] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c reported cstate change: term changed from 0 to 1, leader changed from <none> to f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65). New cstate: current_term: 1 leader_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } health_report { overall_health: UNKNOWN } } }
I20260620 18:44:03.537544 18010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:03.535089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.15.77.65:37547)
0620 18:44:03.535257 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:03.535259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:03.535265 (+ 6us) server_negotiation.cc:407] Connection header received
0620 18:44:03.535282 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:03.535283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:03.535292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:03.535383 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:03.535499 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:03.535997 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:03.536817 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:03.537035 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:03.537118 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:03.537271 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:03.537311 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:03.537370 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:03.537425 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 18:44:03.537451 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260620 18:44:03.539886 17833 raft_consensus.cc:1261] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Refusing update from remote peer f99b7f4a80a8471d885cb4652fefdc6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:03.539954 17693 raft_consensus.cc:1261] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Refusing update from remote peer f99b7f4a80a8471d885cb4652fefdc6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:03.540129 17968 consensus_queue.cc:1035] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:03.540230 17984 consensus_queue.cc:1035] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:03.549770 17395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:03.549898 17395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:03.550566 17395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454: Sending DeleteTablet for 3 replicas of tablet c82b40f11e76466f9c5e8ae4c0877bcf
I20260620 18:44:03.550828 17813 tablet_service.cc:1514] Processing DeleteTablet for tablet c82b40f11e76466f9c5e8ae4c0877bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:03 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:34214
I20260620 18:44:03.550839 17533 tablet_service.cc:1514] Processing DeleteTablet for tablet c82b40f11e76466f9c5e8ae4c0877bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:03 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:53976
I20260620 18:44:03.550947 17992 tablet_replica.cc:330] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad: stopping tablet replica
I20260620 18:44:03.550946 17990 tablet_replica.cc:330] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c: stopping tablet replica
I20260620 18:44:03.550999 17990 raft_consensus.cc:2229] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:44:03.551002 17992 raft_consensus.cc:2229] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:03.551047 17992 raft_consensus.cc:2258] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:03.551052 17990 raft_consensus.cc:2258] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:03.551023 17395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:44:03.551172 17395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:03.551244 17672 tablet_service.cc:1514] Processing DeleteTablet for tablet c82b40f11e76466f9c5e8ae4c0877bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:03 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:47226
I20260620 18:44:03.551358 17991 tablet_replica.cc:330] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b: stopping tablet replica
I20260620 18:44:03.551424 17991 raft_consensus.cc:2229] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:03.551463 17991 raft_consensus.cc:2258] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:03.551462 17992 ts_tablet_manager.cc:1905] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:03.551529 17990 ts_tablet_manager.cc:1905] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:03.552317 17813 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3937c8a6504c128b5764447643d05d (DEFAULT_TABLE table=test-table [id=667559a7446f4899bd151cfd9d8e2c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:03.552316 17533 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3937c8a6504c128b5764447643d05d (DEFAULT_TABLE table=test-table [id=667559a7446f4899bd151cfd9d8e2c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:03.552412 17813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b3937c8a6504c128b5764447643d05d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:03.552462 17533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b3937c8a6504c128b5764447643d05d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:03.553175 17990 ts_tablet_manager.cc:1918] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:03.553233 17990 log.cc:1196] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/c82b40f11e76466f9c5e8ae4c0877bcf
I20260620 18:44:03.553457 17990 ts_tablet_manager.cc:1939] T c82b40f11e76466f9c5e8ae4c0877bcf P f99b7f4a80a8471d885cb4652fefdc6c: Deleting consensus metadata
I20260620 18:44:03.553777 17992 ts_tablet_manager.cc:1918] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:03.553777 18002 tablet_bootstrap.cc:492] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c: Bootstrap starting.
I20260620 18:44:03.553836 17992 log.cc:1196] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/c82b40f11e76466f9c5e8ae4c0877bcf
I20260620 18:44:03.553866 17381 catalog_manager.cc:4841] TS f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65:37547): tablet c82b40f11e76466f9c5e8ae4c0877bcf (table test-table [id=8d4595b9c4c545f2b7f3d71ba4a430b5]) successfully deleted
I20260620 18:44:03.554073 17992 ts_tablet_manager.cc:1939] T c82b40f11e76466f9c5e8ae4c0877bcf P 70753283514f4381bed7e3f3b3e8bbad: Deleting consensus metadata
I20260620 18:44:03.554117 18002 tablet_bootstrap.cc:654] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:03.554358 17991 ts_tablet_manager.cc:1905] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:03.554368 17381 catalog_manager.cc:4841] TS 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855): tablet c82b40f11e76466f9c5e8ae4c0877bcf (table test-table [id=8d4595b9c4c545f2b7f3d71ba4a430b5]) successfully deleted
I20260620 18:44:03.554445 17672 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3937c8a6504c128b5764447643d05d (DEFAULT_TABLE table=test-table [id=667559a7446f4899bd151cfd9d8e2c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:03.554533 17672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b3937c8a6504c128b5764447643d05d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:03.554833 18001 tablet_bootstrap.cc:492] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad: Bootstrap starting.
I20260620 18:44:03.555097 18001 tablet_bootstrap.cc:654] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:03.555663 18001 tablet_bootstrap.cc:492] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad: No bootstrap required, opened a new log
I20260620 18:44:03.555707 18001 ts_tablet_manager.cc:1397] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 18:44:03.555816 18001 raft_consensus.cc:348] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.555891 18001 raft_consensus.cc:374] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:03.555887 17991 ts_tablet_manager.cc:1918] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:03.555924 18001 raft_consensus.cc:729] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70753283514f4381bed7e3f3b3e8bbad, State: Initialized, Role: FOLLOWER
I20260620 18:44:03.555941 17991 log.cc:1196] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/c82b40f11e76466f9c5e8ae4c0877bcf
I20260620 18:44:03.555969 18001 consensus_queue.cc:260] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.556066 18001 ts_tablet_manager.cc:1428] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:03.556190 17991 ts_tablet_manager.cc:1939] T c82b40f11e76466f9c5e8ae4c0877bcf P f84d7b54d43948dbb4634a64af7bbd2b: Deleting consensus metadata
I20260620 18:44:03.556313 18002 tablet_bootstrap.cc:492] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c: No bootstrap required, opened a new log
I20260620 18:44:03.556367 18002 ts_tablet_manager.cc:1397] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 18:44:03.556457 18003 tablet_bootstrap.cc:492] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b: Bootstrap starting.
I20260620 18:44:03.556473 18002 raft_consensus.cc:348] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.556557 18002 raft_consensus.cc:374] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:03.556583 18002 raft_consensus.cc:729] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99b7f4a80a8471d885cb4652fefdc6c, State: Initialized, Role: FOLLOWER
I20260620 18:44:03.556627 18002 consensus_queue.cc:260] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.556726 18002 ts_tablet_manager.cc:1428] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:03.556789 18003 tablet_bootstrap.cc:654] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:03.557080 17380 catalog_manager.cc:4841] TS f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075): tablet c82b40f11e76466f9c5e8ae4c0877bcf (table test-table [id=8d4595b9c4c545f2b7f3d71ba4a430b5]) successfully deleted
I20260620 18:44:03.557333 18003 tablet_bootstrap.cc:492] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b: No bootstrap required, opened a new log
I20260620 18:44:03.557389 18003 ts_tablet_manager.cc:1397] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:03.557523 18003 raft_consensus.cc:348] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.557607 18003 raft_consensus.cc:374] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:03.557637 18003 raft_consensus.cc:729] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84d7b54d43948dbb4634a64af7bbd2b, State: Initialized, Role: FOLLOWER
I20260620 18:44:03.557680 18003 consensus_queue.cc:260] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.557793 18003 ts_tablet_manager.cc:1428] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:03.891429 17968 raft_consensus.cc:482] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:03.891553 17968 raft_consensus.cc:504] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.891727 17968 leader_election.cc:288] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075), 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855)
I20260620 18:44:03.892011 17833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3937c8a6504c128b5764447643d05d" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70753283514f4381bed7e3f3b3e8bbad" is_pre_election: true
I20260620 18:44:03.892011 17693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3937c8a6504c128b5764447643d05d" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" is_pre_election: true
I20260620 18:44:03.892128 17693 raft_consensus.cc:2454] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 0.
I20260620 18:44:03.892134 17833 raft_consensus.cc:2454] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 0.
I20260620 18:44:03.892306 17484 leader_election.cc:302] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f84d7b54d43948dbb4634a64af7bbd2b, f99b7f4a80a8471d885cb4652fefdc6c; no voters:
I20260620 18:44:03.892436 17968 raft_consensus.cc:2781] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:03.892508 17968 raft_consensus.cc:482] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:03.892536 17968 raft_consensus.cc:3037] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:03.893317 17968 raft_consensus.cc:504] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.893472 17968 leader_election.cc:288] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 election: Requested vote from peers f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075), 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855)
I20260620 18:44:03.893630 17833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3937c8a6504c128b5764447643d05d" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70753283514f4381bed7e3f3b3e8bbad"
I20260620 18:44:03.893708 17833 raft_consensus.cc:3037] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:03.893661 17693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3937c8a6504c128b5764447643d05d" candidate_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84d7b54d43948dbb4634a64af7bbd2b"
I20260620 18:44:03.893760 17693 raft_consensus.cc:3037] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:03.894338 17693 raft_consensus.cc:2454] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 1.
I20260620 18:44:03.894361 17833 raft_consensus.cc:2454] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99b7f4a80a8471d885cb4652fefdc6c in term 1.
I20260620 18:44:03.894517 17484 leader_election.cc:302] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f84d7b54d43948dbb4634a64af7bbd2b, f99b7f4a80a8471d885cb4652fefdc6c; no voters:
I20260620 18:44:03.894634 17968 raft_consensus.cc:2781] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:03.894707 17968 raft_consensus.cc:686] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [term 1 LEADER]: Becoming Leader. State: Replica: f99b7f4a80a8471d885cb4652fefdc6c, State: Running, Role: LEADER
I20260620 18:44:03.894749 17968 consensus_queue.cc:237] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } }
I20260620 18:44:03.895397 17396 catalog_manager.cc:5495] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c reported cstate change: term changed from 0 to 1, leader changed from <none> to f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65). New cstate: current_term: 1 leader_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99b7f4a80a8471d885cb4652fefdc6c" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 37547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 } health_report { overall_health: UNKNOWN } } }
I20260620 18:44:03.914409 17833 raft_consensus.cc:1261] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Refusing update from remote peer f99b7f4a80a8471d885cb4652fefdc6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:03.914439 17693 raft_consensus.cc:1261] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Refusing update from remote peer f99b7f4a80a8471d885cb4652fefdc6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:03.914649 17968 consensus_queue.cc:1035] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "70753283514f4381bed7e3f3b3e8bbad" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:03.914755 17981 consensus_queue.cc:1035] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f84d7b54d43948dbb4634a64af7bbd2b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 18:44:03.918977 17973 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 18:44:05.441130 17394 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:05.441260 17394 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:05.442214 17394 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cc296fac101455da43510587f70c454: Sending DeleteTablet for 3 replicas of tablet 9b3937c8a6504c128b5764447643d05d
I20260620 18:44:05.442482 17533 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b3937c8a6504c128b5764447643d05d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:05 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:53976
I20260620 18:44:05.442616 17813 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b3937c8a6504c128b5764447643d05d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:05 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:34214
I20260620 18:44:05.442716 18044 tablet_replica.cc:330] T 9b3937c8a6504c128b5764447643d05d P f99b7f4a80a8471d885cb4652fefdc6c: stopping tablet replica
I20260620 18:44:05.442714 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 17468
I20260620 18:44:05.443583 18045 tablet_replica.cc:330] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad: stopping tablet replica
I20260620 18:44:05.443738 18045 raft_consensus.cc:2229] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:05.443759 17673 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b3937c8a6504c128b5764447643d05d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:05 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:47226
I20260620 18:44:05.443805 18045 raft_consensus.cc:2258] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:05.443943 18046 tablet_replica.cc:330] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b: stopping tablet replica
I20260620 18:44:05.444020 18046 raft_consensus.cc:2229] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:05.444067 18046 raft_consensus.cc:2258] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:05.444083 18045 ts_tablet_manager.cc:1905] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:05.444322 18046 ts_tablet_manager.cc:1905] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:05.445763 18046 ts_tablet_manager.cc:1918] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 18:44:05.445847 18046 log.cc:1196] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/9b3937c8a6504c128b5764447643d05d
I20260620 18:44:05.446079 18046 ts_tablet_manager.cc:1939] T 9b3937c8a6504c128b5764447643d05d P f84d7b54d43948dbb4634a64af7bbd2b: Deleting consensus metadata
I20260620 18:44:05.446120 18045 ts_tablet_manager.cc:1918] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 18:44:05.446224 18045 log.cc:1196] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/9b3937c8a6504c128b5764447643d05d
I20260620 18:44:05.446421 17380 catalog_manager.cc:4841] TS f84d7b54d43948dbb4634a64af7bbd2b (127.15.77.66:37075): tablet 9b3937c8a6504c128b5764447643d05d (table test-table [id=667559a7446f4899bd151cfd9d8e2c74]) successfully deleted
I20260620 18:44:05.446489 18045 ts_tablet_manager.cc:1939] T 9b3937c8a6504c128b5764447643d05d P 70753283514f4381bed7e3f3b3e8bbad: Deleting consensus metadata
I20260620 18:44:05.446810 17381 catalog_manager.cc:4841] TS 70753283514f4381bed7e3f3b3e8bbad (127.15.77.67:39855): tablet 9b3937c8a6504c128b5764447643d05d (table test-table [id=667559a7446f4899bd151cfd9d8e2c74]) successfully deleted
W20260620 18:44:05.450380 17381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:37547 (error 108)
I20260620 18:44:05.450655 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 17608
I20260620 18:44:05.451020 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:05.450811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:37547 (local address 127.0.0.1:48578)
0620 18:44:05.450928 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:05.450972 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:37547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
W20260620 18:44:05.451196 17381 catalog_manager.cc:4584] TS f99b7f4a80a8471d885cb4652fefdc6c (127.15.77.65:37547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9b3937c8a6504c128b5764447643d05d: Network error: Client connection negotiation failed: client connection to 127.15.77.65:37547: connect: Connection refused (error 111)
I20260620 18:44:05.457561 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 17748
I20260620 18:44:05.463550 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 17364
2026-06-20T18:44:05Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6308 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:44:05 dist-test-slave-tvl9 krb5kdc[18051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:44:05 dist-test-slave-tvl9 krb5kdc[18051](info): set up 2 sockets
Jun 20 18:44:05 dist-test-slave-tvl9 krb5kdc[18051](info): commencing operation
krb5kdc: starting...
W20260620 18:44:06.507174 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:44:06 dist-test-slave-tvl9 krb5kdc[18051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:44:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:44:06.630089 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:46065
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:46065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:06.712391 18064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:06.712642 18064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:06.712678 18064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:06.713830 18064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:06.713899 18064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:06.713927 18064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:06.713958 18064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:06.713985 18064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:06.715559 18064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:46065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:46065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:06.715831 18064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:06.716068 18064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:06.721282 18069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:06.721300 18064 server_base.cc:1034] running on GCE node
W20260620 18:44:06.721287 18072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:06.721273 18070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:06.721856 18064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:06.722159 18064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:06.723366 18064 hybrid_clock.cc:648] HybridClock initialized: now 1781981046723359 us; error 38 us; skew 500 ppm
Jun 20 18:44:06 dist-test-slave-tvl9 krb5kdc[18051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981046, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:06.725842 18064 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:44:06.726316 18064 webserver.cc:460] Webserver started at http://127.15.77.126:45257/ using document root <none> and password file <none>
I20260620 18:44:06.726567 18064 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:06.726635 18064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:06.726780 18064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:06.727900 18064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "204bd4b9b62c42c9a96270d40c1c392d"
format_stamp: "Formatted at 2026-06-20 18:44:06 on dist-test-slave-tvl9"
server_key: "0ca4be2632084ad20e2df7ff8f30aaa2"
server_key_iv: "0bf3f021be062f15abae6ca18d22bd8a"
server_key_version: "encryptionkey@0"
I20260620 18:44:06.728256 18064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "204bd4b9b62c42c9a96270d40c1c392d"
format_stamp: "Formatted at 2026-06-20 18:44:06 on dist-test-slave-tvl9"
server_key: "0ca4be2632084ad20e2df7ff8f30aaa2"
server_key_iv: "0bf3f021be062f15abae6ca18d22bd8a"
server_key_version: "encryptionkey@0"
I20260620 18:44:06.729597 18064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:44:06.730280 18079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:06.730444 18064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:06.730554 18064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "204bd4b9b62c42c9a96270d40c1c392d"
format_stamp: "Formatted at 2026-06-20 18:44:06 on dist-test-slave-tvl9"
server_key: "0ca4be2632084ad20e2df7ff8f30aaa2"
server_key_iv: "0bf3f021be062f15abae6ca18d22bd8a"
server_key_version: "encryptionkey@0"
I20260620 18:44:06.730629 18064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:06.742580 18064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:06.745141 18064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:06.745316 18064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:06.749647 18064 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:46065
I20260620 18:44:06.749684 18131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:46065 every 8 connection(s)
I20260620 18:44:06.750082 18064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:06.751003 18132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:06.753393 18132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d: Bootstrap starting.
I20260620 18:44:06.753945 18132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:06.754199 18132 log.cc:824] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:06.754873 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 18064
I20260620 18:44:06.754968 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:06.754972 18132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d: No bootstrap required, opened a new log
I20260620 18:44:06.755154 15669 external_mini_cluster.cc:1428] Setting key 268e940c182260f82407ddd5a51a8088
I20260620 18:44:06.757517 18132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204bd4b9b62c42c9a96270d40c1c392d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46065 } }
I20260620 18:44:06.757653 18132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:06.757676 18132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204bd4b9b62c42c9a96270d40c1c392d, State: Initialized, Role: FOLLOWER
I20260620 18:44:06.757814 18132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204bd4b9b62c42c9a96270d40c1c392d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46065 } }
I20260620 18:44:06.757890 18132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:06.757948 18132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:06.758006 18132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:06.758582 18132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204bd4b9b62c42c9a96270d40c1c392d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46065 } }
I20260620 18:44:06.758697 18132 leader_election.cc:302] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 204bd4b9b62c42c9a96270d40c1c392d; no voters:
I20260620 18:44:06.758898 18132 leader_election.cc:288] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:06.758925 18137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:06.759200 18137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [term 1 LEADER]: Becoming Leader. State: Replica: 204bd4b9b62c42c9a96270d40c1c392d, State: Running, Role: LEADER
I20260620 18:44:06.759372 18132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:06.759389 18137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204bd4b9b62c42c9a96270d40c1c392d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46065 } }
I20260620 18:44:06.759845 18137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 204bd4b9b62c42c9a96270d40c1c392d. Latest consensus state: current_term: 1 leader_uuid: "204bd4b9b62c42c9a96270d40c1c392d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204bd4b9b62c42c9a96270d40c1c392d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46065 } } }
I20260620 18:44:06.759856 18138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "204bd4b9b62c42c9a96270d40c1c392d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "204bd4b9b62c42c9a96270d40c1c392d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46065 } } }
I20260620 18:44:06.760056 18138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:06.760281 18137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:06.760344 18146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:06.760833 18146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:06.762511 18146 catalog_manager.cc:1347] Generated new cluster ID: 5cfa6c28d92d4d13810c1f55f1466ff7
I20260620 18:44:06.762569 18146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:44:06 dist-test-slave-tvl9 krb5kdc[18051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:06.767553 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:06.755710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50656 (local address 127.15.77.126:46065)
0620 18:44:06.755894 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:06.755904 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:06.755942 (+ 38us) server_negotiation.cc:407] Connection header received
0620 18:44:06.756433 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:06.756440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:06.756485 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:06.756598 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:06.757399 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:06.758833 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:06.759977 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:06.760774 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:06.760881 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:06.763765 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:06.763769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:06.763769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:06.763780 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:06.765750 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:06.766176 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:06.766179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:06.766180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:06.766277 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:06.766493 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:06.766493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:06.766494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:06.766788 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:06.766943 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:06.767068 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 18:44:06.767190 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260620 18:44:06.787676 18146 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:06.788371 18146 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:06.794059 18146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 204bd4b9b62c42c9a96270d40c1c392d: Generated new TSK 0
I20260620 18:44:06.794281 18146 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:44:06.821643 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:46065
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:06.905907 18158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:06.906175 18158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:06.906219 18158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:06.907687 18158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:06.907757 18158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:06.907804 18158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:44:06.909344 18158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:46065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:06.909652 18158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:06.909895 18158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:06.915096 18164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:06.915102 18166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:06.915102 18163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:06.915362 18158 server_base.cc:1034] running on GCE node
I20260620 18:44:06.915580 18158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:06.915834 18158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:06.916997 18158 hybrid_clock.cc:648] HybridClock initialized: now 1781981046916985 us; error 29 us; skew 500 ppm
Jun 20 18:44:06 dist-test-slave-tvl9 krb5kdc[18051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981046, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:06.919697 18158 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:44:06.920141 18158 webserver.cc:460] Webserver started at http://127.15.77.65:42707/ using document root <none> and password file <none>
I20260620 18:44:06.920394 18158 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:06.920444 18158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:06.920574 18158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:06.921664 18158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "b952d38be41840d7b3b5f66b5872002b"
format_stamp: "Formatted at 2026-06-20 18:44:06 on dist-test-slave-tvl9"
server_key: "a284a973f130aff309305ec059a78a89"
server_key_iv: "8756ffe3a8873b589ed3ca8b7b240805"
server_key_version: "encryptionkey@0"
I20260620 18:44:06.922032 18158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "b952d38be41840d7b3b5f66b5872002b"
format_stamp: "Formatted at 2026-06-20 18:44:06 on dist-test-slave-tvl9"
server_key: "a284a973f130aff309305ec059a78a89"
server_key_iv: "8756ffe3a8873b589ed3ca8b7b240805"
server_key_version: "encryptionkey@0"
I20260620 18:44:06.923429 18158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 18:44:06.924232 18173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:06.924413 18158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:06.924499 18158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "b952d38be41840d7b3b5f66b5872002b"
format_stamp: "Formatted at 2026-06-20 18:44:06 on dist-test-slave-tvl9"
server_key: "a284a973f130aff309305ec059a78a89"
server_key_iv: "8756ffe3a8873b589ed3ca8b7b240805"
server_key_version: "encryptionkey@0"
I20260620 18:44:06.924587 18158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:06.954624 18158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:06.956657 18158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:06.956779 18158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:06.956990 18158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:06.957334 18158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:06.957368 18158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:06.957391 18158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:06.957444 18158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:06.963762 18158 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:39423
I20260620 18:44:06.963869 18286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:39423 every 8 connection(s)
I20260620 18:44:06.964195 18158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:44:06.967413 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 18158
I20260620 18:44:06.967518 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:44:06.967700 15669 external_mini_cluster.cc:1428] Setting key 88ae8359db1a85d9231a74ea738da0a3
Jun 20 18:44:06 dist-test-slave-tvl9 krb5kdc[18051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981046, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:06.974761 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:06.964993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:48827 (local address 127.15.77.126:46065)
0620 18:44:06.965040 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:06.965042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:06.965263 (+ 221us) server_negotiation.cc:407] Connection header received
0620 18:44:06.965760 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:06.965760 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:06.965770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:06.965835 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:06.966826 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:06.967403 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:06.968942 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:06.969147 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:06.969206 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:06.971725 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:06.971726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:06.971727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:06.971734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:06.973178 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:06.973725 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:06.973725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:06.973726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:06.973777 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:06.974091 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:06.974092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:06.974092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:06.974192 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:06.974243 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:06.974615 (+ 372us) server_negotiation.cc:299] Negotiation successful
0620 18:44:06.974691 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:44:06.975070 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:06.964940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:46065 (local address 127.15.77.65:48827)
0620 18:44:06.965121 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:06.965167 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:06.965659 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:06.965921 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:06.965924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:06.966506 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:06.966730 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:06.966734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:06.967506 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:06.967508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:06.968845 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:06.968849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:06.969239 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:06.969240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:06.969408 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:06.970055 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:06.970065 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:06.971620 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:06.973266 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:06.973276 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:06.973278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:06.973634 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:06.973847 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:06.973848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:06.973848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:06.974020 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:06.974259 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:06.974267 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:06.974463 (+ 196us) client_negotiation.cc:769] Sending connection context
0620 18:44:06.974596 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 18:44:06.974797 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260620 18:44:06.975704 18287 heartbeater.cc:347] Connected to a master server at 127.15.77.126:46065
I20260620 18:44:06.975819 18287 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:06.976038 18287 heartbeater.cc:511] Master 127.15.77.126:46065 requested a full tablet report, sending...
I20260620 18:44:06.976621 18096 ts_manager.cc:194] Registered new tserver with Master: b952d38be41840d7b3b5f66b5872002b (127.15.77.65:39423)
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:44:06.977339 18096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:48827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:44:06.994539 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:46065
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:07.078267 18292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:07.078503 18292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:07.078531 18292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:07.079783 18292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:07.079830 18292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:07.079857 18292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:44:07.081310 18292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:46065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:07.081523 18292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:07.081720 18292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:07.086906 18297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:07.086948 18300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:07.087010 18298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:07.087189 18292 server_base.cc:1034] running on GCE node
I20260620 18:44:07.087440 18292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:07.087666 18292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:07.088853 18292 hybrid_clock.cc:648] HybridClock initialized: now 1781981047088849 us; error 37 us; skew 500 ppm
Jun 20 18:44:07 dist-test-slave-tvl9 krb5kdc[18051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981047, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:07.091580 18292 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:44:07.092020 18292 webserver.cc:460] Webserver started at http://127.15.77.66:38261/ using document root <none> and password file <none>
I20260620 18:44:07.092272 18292 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:07.092343 18292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:07.092491 18292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:07.093559 18292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "5be25986a3904b0fafecf18e7329bd21"
format_stamp: "Formatted at 2026-06-20 18:44:07 on dist-test-slave-tvl9"
server_key: "bb347bbad950a7437ef7c74e1a87625a"
server_key_iv: "03d5eecff29c5470b73c317847be9e97"
server_key_version: "encryptionkey@0"
I20260620 18:44:07.093919 18292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "5be25986a3904b0fafecf18e7329bd21"
format_stamp: "Formatted at 2026-06-20 18:44:07 on dist-test-slave-tvl9"
server_key: "bb347bbad950a7437ef7c74e1a87625a"
server_key_iv: "03d5eecff29c5470b73c317847be9e97"
server_key_version: "encryptionkey@0"
I20260620 18:44:07.095234 18292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:44:07.096127 18307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:07.096309 18292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:07.096426 18292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "5be25986a3904b0fafecf18e7329bd21"
format_stamp: "Formatted at 2026-06-20 18:44:07 on dist-test-slave-tvl9"
server_key: "bb347bbad950a7437ef7c74e1a87625a"
server_key_iv: "03d5eecff29c5470b73c317847be9e97"
server_key_version: "encryptionkey@0"
I20260620 18:44:07.096520 18292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:07.110481 18292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:07.112679 18292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:07.112849 18292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:07.113097 18292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:07.113410 18292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:07.113473 18292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:07.113546 18292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:07.113579 18292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:07.120074 18292 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:44405
I20260620 18:44:07.120137 18420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:44405 every 8 connection(s)
I20260620 18:44:07.120527 18292 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
Jun 20 18:44:07 dist-test-slave-tvl9 krb5kdc[18051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981047, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:07.129678 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 18292
I20260620 18:44:07.129765 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:44:07.129990 15669 external_mini_cluster.cc:1428] Setting key 911e5190f37a8d6954dded6430ad4870
I20260620 18:44:07.130359 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:07.121297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:34871 (local address 127.15.77.126:46065)
0620 18:44:07.121366 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:07.121368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:07.121590 (+ 222us) server_negotiation.cc:407] Connection header received
0620 18:44:07.122140 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:07.122141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:07.122154 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:07.122210 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:07.123195 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:07.123774 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:07.124801 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:07.125012 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:07.125103 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:07.127419 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:07.127421 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:07.127421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:07.127429 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:07.128870 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:07.129304 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:07.129306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:07.129306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:07.129368 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:07.129634 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:07.129636 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:07.129636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:07.129784 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:07.129850 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:07.130180 (+ 330us) server_negotiation.cc:299] Negotiation successful
0620 18:44:07.130268 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 18:44:07.130569 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:07.121282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:46065 (local address 127.15.77.66:34871)
0620 18:44:07.121448 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:07.121486 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:07.122029 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:07.122289 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:07.122293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:07.122870 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:07.123082 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:07.123087 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:07.123868 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:07.123870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:07.124691 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:07.124693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:07.125112 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:07.125113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:07.125248 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:07.125793 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:07.125801 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:07.127271 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:07.128970 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:07.128978 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:07.128980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:07.129212 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:07.129436 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:07.129437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:07.129437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:07.129565 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:07.129883 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:07.129889 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:07.130011 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 18:44:07.130144 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 18:44:07.130308 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260620 18:44:07.130995 18421 heartbeater.cc:347] Connected to a master server at 127.15.77.126:46065
I20260620 18:44:07.131110 18421 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:07.131295 18421 heartbeater.cc:511] Master 127.15.77.126:46065 requested a full tablet report, sending...
I20260620 18:44:07.131758 18096 ts_manager.cc:194] Registered new tserver with Master: 5be25986a3904b0fafecf18e7329bd21 (127.15.77.66:44405)
I20260620 18:44:07.132277 18096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:34871
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:44:07.154132 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:46065
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:07.239887 18426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:07.240108 18426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:07.240132 18426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:07.241266 18426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:07.241302 18426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:07.241328 18426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:44:07.242715 18426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:46065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:07.242911 18426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:07.243100 18426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:07.248248 18434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:07.248266 18432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:07.248270 18431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:07.248459 18426 server_base.cc:1034] running on GCE node
I20260620 18:44:07.248800 18426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:07.249007 18426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:07.250159 18426 hybrid_clock.cc:648] HybridClock initialized: now 1781981047250152 us; error 31 us; skew 500 ppm
Jun 20 18:44:07 dist-test-slave-tvl9 krb5kdc[18051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981047, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:07.252820 18426 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:44:07.253262 18426 webserver.cc:460] Webserver started at http://127.15.77.67:43107/ using document root <none> and password file <none>
I20260620 18:44:07.253492 18426 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:07.253561 18426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:07.253710 18426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:07.254757 18426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "b4215b83e8074810b7b2efe6aa28c8e4"
format_stamp: "Formatted at 2026-06-20 18:44:07 on dist-test-slave-tvl9"
server_key: "5a60d28946636ed92c694be6288ef3fe"
server_key_iv: "fc7eb980f6055d4d49c47da5b65ce799"
server_key_version: "encryptionkey@0"
I20260620 18:44:07.255123 18426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "b4215b83e8074810b7b2efe6aa28c8e4"
format_stamp: "Formatted at 2026-06-20 18:44:07 on dist-test-slave-tvl9"
server_key: "5a60d28946636ed92c694be6288ef3fe"
server_key_iv: "fc7eb980f6055d4d49c47da5b65ce799"
server_key_version: "encryptionkey@0"
I20260620 18:44:07.256512 18426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:07.257328 18441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:07.257508 18426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 18:44:07.257611 18426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "b4215b83e8074810b7b2efe6aa28c8e4"
format_stamp: "Formatted at 2026-06-20 18:44:07 on dist-test-slave-tvl9"
server_key: "5a60d28946636ed92c694be6288ef3fe"
server_key_iv: "fc7eb980f6055d4d49c47da5b65ce799"
server_key_version: "encryptionkey@0"
I20260620 18:44:07.257694 18426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:07.271756 18426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:07.274143 18426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:07.274307 18426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:07.274540 18426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:07.274855 18426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:07.274905 18426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:07.274943 18426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:07.274971 18426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:07.280833 18426 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:38337
I20260620 18:44:07.280879 18554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:38337 every 8 connection(s)
I20260620 18:44:07.281224 18426 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
Jun 20 18:44:07 dist-test-slave-tvl9 krb5kdc[18051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981047, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:07.289119 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 18426
I20260620 18:44:07.289233 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:44:07.289462 15669 external_mini_cluster.cc:1428] Setting key 704af8a36c4944f3064361cc02a4d9d4
I20260620 18:44:07.290863 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:07.281949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:57289 (local address 127.15.77.126:46065)
0620 18:44:07.282012 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:07.282014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:07.282206 (+ 192us) server_negotiation.cc:407] Connection header received
0620 18:44:07.282677 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:07.282678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:07.282688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:07.282751 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:07.283816 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:07.284311 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:07.285313 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:07.285514 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:07.285578 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:07.287739 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:07.287740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:07.287741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:07.287747 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:07.289206 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:07.289854 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:07.289855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:07.289855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:07.289920 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:07.290214 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:07.290215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:07.290215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:07.290379 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:07.290446 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:07.290730 (+ 284us) server_negotiation.cc:299] Negotiation successful
0620 18:44:07.290799 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 18:44:07.290961 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:07.281888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:46065 (local address 127.15.77.67:57289)
0620 18:44:07.282064 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:07.282110 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:07.282590 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:07.282822 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:07.282826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:07.283516 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:07.283724 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:07.283729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:07.284388 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:07.284389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:07.285205 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:07.285207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:07.285591 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:07.285592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:07.285693 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:07.286155 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:07.286163 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:07.287647 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:07.289338 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:07.289345 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:07.289348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:07.289781 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:07.290011 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:07.290012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:07.290013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:07.290149 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:07.290447 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:07.290451 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:07.290584 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 18:44:07.290675 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 18:44:07.290799 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260620 18:44:07.291458 18555 heartbeater.cc:347] Connected to a master server at 127.15.77.126:46065
I20260620 18:44:07.291615 18555 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:07.291769 18555 heartbeater.cc:511] Master 127.15.77.126:46065 requested a full tablet report, sending...
I20260620 18:44:07.292137 18096 ts_manager.cc:194] Registered new tserver with Master: b4215b83e8074810b7b2efe6aa28c8e4 (127.15.77.67:38337)
I20260620 18:44:07.292523 18096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:57289
I20260620 18:44:07.301301 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 18:44:07.313048 18136 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:44:07.311957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50662 (local address 127.15.77.126:46065)
0620 18:44:07.312046 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:07.312048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:07.312159 (+ 111us) server_negotiation.cc:407] Connection header received
0620 18:44:07.312260 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:07.312261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:07.312272 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:07.312344 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:07.312963 (+ 619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50662: BlockingRecv error: recv got EOF from 127.0.0.1:50662 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 18:44:07.313925 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 18158
I20260620 18:44:07.318660 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 18292
I20260620 18:44:07.322990 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 18426
I20260620 18:44:07.327428 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 18064
2026-06-20T18:44:07Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1862 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:44:07 dist-test-slave-tvl9 krb5kdc[18567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:44:07 dist-test-slave-tvl9 krb5kdc[18567](info): set up 2 sockets
Jun 20 18:44:07 dist-test-slave-tvl9 krb5kdc[18567](info): commencing operation
krb5kdc: starting...
W20260620 18:44:08.359182 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:44:08 dist-test-slave-tvl9 krb5kdc[18567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:44:08.480036 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:35415
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:41189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:35415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:08.563192 18580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:08.563439 18580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:08.563469 18580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:08.564615 18580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:08.564652 18580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:08.564667 18580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:08.564679 18580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:08.564692 18580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:08.566159 18580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:41189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:35415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:35415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:08.566370 18580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:08.566545 18580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:08.571396 18585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:08.571355 18586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:08.571368 18588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:08.571666 18580 server_base.cc:1034] running on GCE node
I20260620 18:44:08.572032 18580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:08.572299 18580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:08.573441 18580 hybrid_clock.cc:648] HybridClock initialized: now 1781981048573427 us; error 30 us; skew 500 ppm
Jun 20 18:44:08 dist-test-slave-tvl9 krb5kdc[18567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981048, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:08.575922 18580 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:44:08.576327 18580 webserver.cc:460] Webserver started at http://127.15.77.126:35409/ using document root <none> and password file <none>
I20260620 18:44:08.576566 18580 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:08.576637 18580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:08.576792 18580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:08.577903 18580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "b972e58558bd4316a11065a3fbd6a917"
format_stamp: "Formatted at 2026-06-20 18:44:08 on dist-test-slave-tvl9"
server_key: "fdf39421dd83cfe9b992c536a91f7193"
server_key_iv: "be35be40852b57b0fee8dc03e9a746d2"
server_key_version: "encryptionkey@0"
I20260620 18:44:08.578269 18580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "b972e58558bd4316a11065a3fbd6a917"
format_stamp: "Formatted at 2026-06-20 18:44:08 on dist-test-slave-tvl9"
server_key: "fdf39421dd83cfe9b992c536a91f7193"
server_key_iv: "be35be40852b57b0fee8dc03e9a746d2"
server_key_version: "encryptionkey@0"
I20260620 18:44:08.579655 18580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 18:44:08.580379 18595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:08.580564 18580 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:08.580672 18580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "b972e58558bd4316a11065a3fbd6a917"
format_stamp: "Formatted at 2026-06-20 18:44:08 on dist-test-slave-tvl9"
server_key: "fdf39421dd83cfe9b992c536a91f7193"
server_key_iv: "be35be40852b57b0fee8dc03e9a746d2"
server_key_version: "encryptionkey@0"
I20260620 18:44:08.580746 18580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:08.593338 18580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:08.595520 18580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:08.595698 18580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:08.599722 18580 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:35415
I20260620 18:44:08.599758 18647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:35415 every 8 connection(s)
I20260620 18:44:08.600270 18580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:08.601009 18648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:08.603427 18648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917: Bootstrap starting.
I20260620 18:44:08.604056 18648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:08.604370 18648 log.cc:824] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:08.604986 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 18580
I20260620 18:44:08.605080 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:08.605124 18648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917: No bootstrap required, opened a new log
I20260620 18:44:08.605269 15669 external_mini_cluster.cc:1428] Setting key d7d9be0bf7a9e5c393b8ef1c83355bb9
I20260620 18:44:08.608124 18648 raft_consensus.cc:348] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e58558bd4316a11065a3fbd6a917" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35415 } }
I20260620 18:44:08.608295 18648 raft_consensus.cc:374] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:08.608328 18648 raft_consensus.cc:729] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b972e58558bd4316a11065a3fbd6a917, State: Initialized, Role: FOLLOWER
I20260620 18:44:08.608448 18648 consensus_queue.cc:260] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e58558bd4316a11065a3fbd6a917" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35415 } }
I20260620 18:44:08.608525 18648 raft_consensus.cc:388] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:08.608552 18648 raft_consensus.cc:482] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:08.608594 18648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:08.609269 18648 raft_consensus.cc:504] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e58558bd4316a11065a3fbd6a917" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35415 } }
I20260620 18:44:08.609359 18648 leader_election.cc:302] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b972e58558bd4316a11065a3fbd6a917; no voters:
I20260620 18:44:08.609542 18648 leader_election.cc:288] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:08.609570 18653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:08.609680 18653 raft_consensus.cc:686] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [term 1 LEADER]: Becoming Leader. State: Replica: b972e58558bd4316a11065a3fbd6a917, State: Running, Role: LEADER
I20260620 18:44:08.609762 18653 consensus_queue.cc:237] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e58558bd4316a11065a3fbd6a917" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35415 } }
I20260620 18:44:08.609807 18648 sys_catalog.cc:564] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:08.610088 18653 sys_catalog.cc:455] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b972e58558bd4316a11065a3fbd6a917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e58558bd4316a11065a3fbd6a917" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35415 } } }
I20260620 18:44:08.610349 18653 sys_catalog.cc:458] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:08.610574 18653 sys_catalog.cc:455] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b972e58558bd4316a11065a3fbd6a917. Latest consensus state: current_term: 1 leader_uuid: "b972e58558bd4316a11065a3fbd6a917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e58558bd4316a11065a3fbd6a917" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35415 } } }
I20260620 18:44:08.610651 18662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:08.610692 18653 sys_catalog.cc:458] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:08.611244 18662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 18:44:08 dist-test-slave-tvl9 krb5kdc[18567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:08.613197 18662 catalog_manager.cc:1347] Generated new cluster ID: a4e9a76586e6480eba74b47322515a4b
I20260620 18:44:08.613251 18662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:44:08.616871 18652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:08.605777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35852 (local address 127.15.77.126:35415)
0620 18:44:08.606006 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:08.606016 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:08.606050 (+ 34us) server_negotiation.cc:407] Connection header received
0620 18:44:08.606660 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:08.606669 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:08.606736 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:08.606903 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:08.607568 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:08.608968 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:08.609937 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:08.610577 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:08.610699 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:08.613595 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:08.613599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:08.613599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:08.613618 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:08.615287 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:08.615735 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:08.615738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:08.615738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:08.615841 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:08.616057 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:08.616057 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:08.616058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:08.616291 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:08.616467 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:08.616532 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 18:44:08.616682 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260620 18:44:08.617763 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:44:08.636577 18662 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:08.637205 18662 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:08.637605 18652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:08.632791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35854 (local address 127.15.77.126:35415)
0620 18:44:08.632837 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:08.632839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:08.632952 (+ 113us) server_negotiation.cc:407] Connection header received
0620 18:44:08.633022 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:08.633023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:08.633035 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:08.633112 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:08.633545 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:08.634348 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:08.635427 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:08.636203 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:08.636279 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:08.637047 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:08.637049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:08.637049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:08.637315 (+ 266us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:08.637336 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:08.637343 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:08.637401 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:08.637484 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 18:44:08.637531 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:44:08.654280 18662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b972e58558bd4316a11065a3fbd6a917: Generated new TSK 0
I20260620 18:44:08.654516 18662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:44:08.660476 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 18580
2026-06-20T18:44:08Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1330 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:44:08 dist-test-slave-tvl9 krb5kdc[18681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:44:08 dist-test-slave-tvl9 krb5kdc[18681](info): set up 2 sockets
Jun 20 18:44:08 dist-test-slave-tvl9 krb5kdc[18681](info): commencing operation
krb5kdc: starting...
W20260620 18:44:09.727408 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:44:09 dist-test-slave-tvl9 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:44:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:44:09.849854 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:45247
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:42475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:45247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:09.933490 18694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:09.933770 18694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:09.933812 18694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:09.935101 18694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:09.935173 18694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:09.935196 18694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:09.935218 18694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:09.935247 18694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:09.937219 18694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:42475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:45247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:45247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:09.937532 18694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:09.937795 18694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:09.942708 18700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:09.942809 18694 server_base.cc:1034] running on GCE node
W20260620 18:44:09.942746 18699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:09.942751 18702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:09.943246 18694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:09.943607 18694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:09.944768 18694 hybrid_clock.cc:648] HybridClock initialized: now 1781981049944749 us; error 40 us; skew 500 ppm
Jun 20 18:44:09 dist-test-slave-tvl9 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981049, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:09.947355 18694 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:44:09.947788 18694 webserver.cc:460] Webserver started at http://127.15.77.126:38173/ using document root <none> and password file <none>
I20260620 18:44:09.948052 18694 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:09.948124 18694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:09.948282 18694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:09.949383 18694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "e0e5cf5b3f534016b2ddcd6d163f300d"
format_stamp: "Formatted at 2026-06-20 18:44:09 on dist-test-slave-tvl9"
server_key: "547088d882d8d92661af30a28ffe8457"
server_key_iv: "b09b15099571ece031f50d14199e56a2"
server_key_version: "encryptionkey@0"
I20260620 18:44:09.949759 18694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "e0e5cf5b3f534016b2ddcd6d163f300d"
format_stamp: "Formatted at 2026-06-20 18:44:09 on dist-test-slave-tvl9"
server_key: "547088d882d8d92661af30a28ffe8457"
server_key_iv: "b09b15099571ece031f50d14199e56a2"
server_key_version: "encryptionkey@0"
I20260620 18:44:09.951169 18694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:09.952059 18709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:09.952283 18694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:09.952389 18694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "e0e5cf5b3f534016b2ddcd6d163f300d"
format_stamp: "Formatted at 2026-06-20 18:44:09 on dist-test-slave-tvl9"
server_key: "547088d882d8d92661af30a28ffe8457"
server_key_iv: "b09b15099571ece031f50d14199e56a2"
server_key_version: "encryptionkey@0"
I20260620 18:44:09.952474 18694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:09.970808 18694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:09.972992 18694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:09.973168 18694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:09.977327 18694 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:45247
I20260620 18:44:09.977365 18761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:45247 every 8 connection(s)
I20260620 18:44:09.977766 18694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:09.978789 18762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:09.981082 18762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d: Bootstrap starting.
I20260620 18:44:09.981693 18762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:09.982002 18762 log.cc:824] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:09.982723 18762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d: No bootstrap required, opened a new log
I20260620 18:44:09.984368 18762 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e5cf5b3f534016b2ddcd6d163f300d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45247 } }
I20260620 18:44:09.984511 18762 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:09.984534 18762 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e5cf5b3f534016b2ddcd6d163f300d, State: Initialized, Role: FOLLOWER
I20260620 18:44:09.984716 18762 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e5cf5b3f534016b2ddcd6d163f300d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45247 } }
I20260620 18:44:09.984797 18762 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:09.984838 18762 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:09.984889 18762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:09.985291 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 18694
I20260620 18:44:09.985396 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:09.985610 15669 external_mini_cluster.cc:1428] Setting key 7e5aa2f2a8f2f30c4b851a88a5d4ae7d
I20260620 18:44:09.986052 18762 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e5cf5b3f534016b2ddcd6d163f300d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45247 } }
I20260620 18:44:09.986205 18762 leader_election.cc:302] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0e5cf5b3f534016b2ddcd6d163f300d; no voters:
I20260620 18:44:09.986434 18762 leader_election.cc:288] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:09.986479 18767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:09.986666 18767 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [term 1 LEADER]: Becoming Leader. State: Replica: e0e5cf5b3f534016b2ddcd6d163f300d, State: Running, Role: LEADER
I20260620 18:44:09.986743 18762 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:09.986811 18767 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e5cf5b3f534016b2ddcd6d163f300d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45247 } }
I20260620 18:44:09.987222 18767 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0e5cf5b3f534016b2ddcd6d163f300d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e5cf5b3f534016b2ddcd6d163f300d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45247 } } }
I20260620 18:44:09.987301 18767 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:09.987514 18767 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0e5cf5b3f534016b2ddcd6d163f300d. Latest consensus state: current_term: 1 leader_uuid: "e0e5cf5b3f534016b2ddcd6d163f300d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e5cf5b3f534016b2ddcd6d163f300d" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45247 } } }
I20260620 18:44:09.987632 18767 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:09.987994 18774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:09.988534 18774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:09.990379 18774 catalog_manager.cc:1347] Generated new cluster ID: 714717fa149144ac832e451f624e8e2c
I20260620 18:44:09.990437 18774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:44:09 dist-test-slave-tvl9 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:09.997764 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:09.986132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58946 (local address 127.15.77.126:45247)
0620 18:44:09.986312 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:09.986321 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:09.986348 (+ 27us) server_negotiation.cc:407] Connection header received
0620 18:44:09.986476 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:09.986484 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:09.986559 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:09.986721 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:09.987618 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:09.989180 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:09.990204 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:09.990821 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:09.991211 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:09.993646 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:09.993650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:09.993651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:09.993663 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:09.995718 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:09.996113 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:09.996116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:09.996116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:09.996224 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:09.996413 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:09.996414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:09.996414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:09.996717 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:09.997290 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:09.997361 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 18:44:09.997513 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260620 18:44:10.023729 18774 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:10.024334 18774 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:10.028925 18774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0e5cf5b3f534016b2ddcd6d163f300d: Generated new TSK 0
I20260620 18:44:10.029134 18774 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:44:10.059213 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45247
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:42475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:10.157480 18788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:10.157724 18788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:10.157764 18788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:10.158927 18788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:10.158999 18788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:10.159044 18788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:44:10.160679 18788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:42475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:10.160953 18788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:10.161193 18788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:10.166364 18794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:10.166397 18796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:10.166397 18793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:10.166540 18788 server_base.cc:1034] running on GCE node
I20260620 18:44:10.166855 18788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:10.167109 18788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:10.168285 18788 hybrid_clock.cc:648] HybridClock initialized: now 1781981050168262 us; error 36 us; skew 500 ppm
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981050, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:10.171006 18788 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:44:10.171586 18788 webserver.cc:460] Webserver started at http://127.15.77.65:35057/ using document root <none> and password file <none>
I20260620 18:44:10.171809 18788 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:10.171851 18788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:10.171943 18788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:10.173032 18788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "72a27c0d7d3e440dab4156f1ec289357"
format_stamp: "Formatted at 2026-06-20 18:44:10 on dist-test-slave-tvl9"
server_key: "4f1ba04980a0d1b3128d287c20e34339"
server_key_iv: "51c1067f3d478594d5298d6d7c46da38"
server_key_version: "encryptionkey@0"
I20260620 18:44:10.173358 18788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "72a27c0d7d3e440dab4156f1ec289357"
format_stamp: "Formatted at 2026-06-20 18:44:10 on dist-test-slave-tvl9"
server_key: "4f1ba04980a0d1b3128d287c20e34339"
server_key_iv: "51c1067f3d478594d5298d6d7c46da38"
server_key_version: "encryptionkey@0"
I20260620 18:44:10.174669 18788 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:10.175451 18803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:10.175613 18788 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:10.175709 18788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "72a27c0d7d3e440dab4156f1ec289357"
format_stamp: "Formatted at 2026-06-20 18:44:10 on dist-test-slave-tvl9"
server_key: "4f1ba04980a0d1b3128d287c20e34339"
server_key_iv: "51c1067f3d478594d5298d6d7c46da38"
server_key_version: "encryptionkey@0"
I20260620 18:44:10.175768 18788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:10.190656 18788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:10.193164 18788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:10.193279 18788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:10.193468 18788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:10.193822 18788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:10.193861 18788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:10.193885 18788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:10.193938 18788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:10.199697 18788 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:33297
I20260620 18:44:10.199826 18916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:33297 every 8 connection(s)
I20260620 18:44:10.200119 18788 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:44:10.205281 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 18788
I20260620 18:44:10.205364 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:44:10.205598 15669 external_mini_cluster.cc:1428] Setting key 65318a63aa8afb9938a702560ac96913
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981050, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:10.210371 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:10.200950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:39883 (local address 127.15.77.126:45247)
0620 18:44:10.201029 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:10.201031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:10.201274 (+ 243us) server_negotiation.cc:407] Connection header received
0620 18:44:10.201747 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:10.201748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:10.201763 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:10.201818 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:10.202783 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.203311 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.204380 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.204641 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.204708 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:10.207160 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:10.207162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:10.207162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:10.207169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:10.208830 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:10.209342 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:10.209343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:10.209343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:10.209402 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:10.209696 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:10.209696 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:10.209697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:10.209804 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:10.209856 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:10.210219 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 18:44:10.210303 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 18:44:10.210707 18919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:10.200926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:45247 (local address 127.15.77.65:39883)
0620 18:44:10.201143 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:10.201181 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:10.201659 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:10.201890 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:10.201893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:10.202464 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:10.202674 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:10.202679 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.203401 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.203403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:10.204265 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:10.204268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.204733 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.204734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:10.204851 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:10.205467 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:10.205478 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:10.207059 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:10.208935 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:10.208943 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:10.208946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:10.209261 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:10.209474 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:10.209475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:10.209475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:10.209631 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:10.209868 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:10.209874 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:10.210051 (+ 177us) client_negotiation.cc:769] Sending connection context
0620 18:44:10.210198 (+ 147us) client_negotiation.cc:240] Negotiation successful
0620 18:44:10.210391 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":124,"threads_started":1}
I20260620 18:44:10.211265 18917 heartbeater.cc:347] Connected to a master server at 127.15.77.126:45247
I20260620 18:44:10.211396 18917 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:10.211625 18917 heartbeater.cc:511] Master 127.15.77.126:45247 requested a full tablet report, sending...
I20260620 18:44:10.212172 18726 ts_manager.cc:194] Registered new tserver with Master: 72a27c0d7d3e440dab4156f1ec289357 (127.15.77.65:33297)
I20260620 18:44:10.212805 18726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:39883
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:44:10.238837 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45247
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:42475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:10.323499 18922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:10.323822 18922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:10.323866 18922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:10.325897 18922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:10.325965 18922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:10.326009 18922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:44:10.328341 18922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:42475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:10.328660 18922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:10.328912 18922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:10.334029 18927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:10.334084 18928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:10.334239 18922 server_base.cc:1034] running on GCE node
W20260620 18:44:10.334344 18930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:10.334546 18922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:10.334757 18922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:10.335919 18922 hybrid_clock.cc:648] HybridClock initialized: now 1781981050335902 us; error 32 us; skew 500 ppm
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981050, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:10.338529 18922 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:44:10.339020 18922 webserver.cc:460] Webserver started at http://127.15.77.66:36179/ using document root <none> and password file <none>
I20260620 18:44:10.339265 18922 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:10.339383 18922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:10.339543 18922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:10.340597 18922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "a86bc5fcb8d24806911c58ab79680acd"
format_stamp: "Formatted at 2026-06-20 18:44:10 on dist-test-slave-tvl9"
server_key: "befb547d3153181df49769860e9aba24"
server_key_iv: "b03e786189a0d492c6e2b0b643b5afec"
server_key_version: "encryptionkey@0"
I20260620 18:44:10.340919 18922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "a86bc5fcb8d24806911c58ab79680acd"
format_stamp: "Formatted at 2026-06-20 18:44:10 on dist-test-slave-tvl9"
server_key: "befb547d3153181df49769860e9aba24"
server_key_iv: "b03e786189a0d492c6e2b0b643b5afec"
server_key_version: "encryptionkey@0"
I20260620 18:44:10.342329 18922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:44:10.343174 18937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:10.343420 18922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:10.343591 18922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "a86bc5fcb8d24806911c58ab79680acd"
format_stamp: "Formatted at 2026-06-20 18:44:10 on dist-test-slave-tvl9"
server_key: "befb547d3153181df49769860e9aba24"
server_key_iv: "b03e786189a0d492c6e2b0b643b5afec"
server_key_version: "encryptionkey@0"
I20260620 18:44:10.343680 18922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:10.364715 18922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:10.366880 18922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:10.367070 18922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:10.367360 18922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:10.367712 18922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:10.367769 18922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:10.367805 18922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:10.367833 18922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:10.374202 18922 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:38889
I20260620 18:44:10.374255 19050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:38889 every 8 connection(s)
I20260620 18:44:10.374621 18922 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981050, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:10.384255 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 18922
I20260620 18:44:10.384394 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:44:10.384816 15669 external_mini_cluster.cc:1428] Setting key 94d17e571b793237debd43ac24b0900e
I20260620 18:44:10.384948 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:10.375427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:35141 (local address 127.15.77.126:45247)
0620 18:44:10.375498 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:10.375500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:10.375727 (+ 227us) server_negotiation.cc:407] Connection header received
0620 18:44:10.376289 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:10.376290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:10.376304 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:10.376364 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:10.377412 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.377934 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.379057 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.379266 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.379366 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:10.381660 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:10.381661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:10.381662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:10.381669 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:10.383166 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:10.383650 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:10.383650 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:10.383651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:10.383702 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:10.383988 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:10.383990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:10.383990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:10.384209 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:10.384297 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:10.384757 (+ 460us) server_negotiation.cc:299] Negotiation successful
0620 18:44:10.384855 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 18:44:10.385138 19053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:10.375442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:45247 (local address 127.15.77.66:35141)
0620 18:44:10.375598 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:10.375640 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:10.376167 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:10.376446 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:10.376450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:10.377090 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:10.377297 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:10.377302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.378048 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.378049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:10.378960 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:10.378962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.379376 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.379378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:10.379530 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:10.380136 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:10.380145 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:10.381537 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:10.383245 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:10.383251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:10.383253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:10.383583 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:10.383762 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:10.383763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:10.383763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:10.383864 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:10.384297 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:10.384302 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:10.384539 (+ 237us) client_negotiation.cc:769] Sending connection context
0620 18:44:10.384688 (+ 149us) client_negotiation.cc:240] Negotiation successful
0620 18:44:10.384845 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260620 18:44:10.385717 19051 heartbeater.cc:347] Connected to a master server at 127.15.77.126:45247
I20260620 18:44:10.385814 19051 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:10.386010 19051 heartbeater.cc:511] Master 127.15.77.126:45247 requested a full tablet report, sending...
I20260620 18:44:10.386368 18726 ts_manager.cc:194] Registered new tserver with Master: a86bc5fcb8d24806911c58ab79680acd (127.15.77.66:38889)
I20260620 18:44:10.386778 18726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:35141
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:44:10.410817 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45247
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:42475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:10.493547 19056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:10.493803 19056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:10.493844 19056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:10.495031 19056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:10.495101 19056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:10.495149 19056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:44:10.496696 19056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:42475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:10.496981 19056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:10.497221 19056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:10.502452 19061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:10.502452 19062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:10.502487 19064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:10.502482 19056 server_base.cc:1034] running on GCE node
I20260620 18:44:10.502960 19056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:10.503206 19056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:10.504370 19056 hybrid_clock.cc:648] HybridClock initialized: now 1781981050504365 us; error 37 us; skew 500 ppm
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981050, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:10.506858 19056 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:44:10.507304 19056 webserver.cc:460] Webserver started at http://127.15.77.67:33705/ using document root <none> and password file <none>
I20260620 18:44:10.507601 19056 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:10.507679 19056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:10.507825 19056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:10.508895 19056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "deef129257544d439a11352d60f5cedf"
format_stamp: "Formatted at 2026-06-20 18:44:10 on dist-test-slave-tvl9"
server_key: "ec7d67ff0877edf1fce29ce56b2b2ef3"
server_key_iv: "fa4d183b19994794f12dca322105fa28"
server_key_version: "encryptionkey@0"
I20260620 18:44:10.509256 19056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "deef129257544d439a11352d60f5cedf"
format_stamp: "Formatted at 2026-06-20 18:44:10 on dist-test-slave-tvl9"
server_key: "ec7d67ff0877edf1fce29ce56b2b2ef3"
server_key_iv: "fa4d183b19994794f12dca322105fa28"
server_key_version: "encryptionkey@0"
I20260620 18:44:10.510603 19056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:44:10.511399 19071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:10.511601 19056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:10.511703 19056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "deef129257544d439a11352d60f5cedf"
format_stamp: "Formatted at 2026-06-20 18:44:10 on dist-test-slave-tvl9"
server_key: "ec7d67ff0877edf1fce29ce56b2b2ef3"
server_key_iv: "fa4d183b19994794f12dca322105fa28"
server_key_version: "encryptionkey@0"
I20260620 18:44:10.511786 19056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:10.533336 19056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:10.535616 19056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:10.535764 19056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:10.535954 19056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:10.536226 19056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:10.536257 19056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:10.536279 19056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:10.536293 19056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:10.542183 19056 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:39239
I20260620 18:44:10.542260 19184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:39239 every 8 connection(s)
I20260620 18:44:10.542603 19056 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:44:10.545928 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 19056
I20260620 18:44:10.546025 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:44:10.546200 15669 external_mini_cluster.cc:1428] Setting key c6574dd5225dc7dbd6c8b6cf410104d9
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981050, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:10.552229 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:10.543397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:39163 (local address 127.15.77.126:45247)
0620 18:44:10.543467 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:10.543469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:10.543684 (+ 215us) server_negotiation.cc:407] Connection header received
0620 18:44:10.544164 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:10.544165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:10.544175 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:10.544226 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:10.545128 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.545620 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.547185 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.547446 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.547502 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:10.549590 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:10.549592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:10.549592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:10.549598 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:10.551030 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:10.551442 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:10.551442 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:10.551443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:10.551493 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:10.551716 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:10.551716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:10.551717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:10.551821 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:10.551872 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:10.552101 (+ 229us) server_negotiation.cc:299] Negotiation successful
0620 18:44:10.552164 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 18:44:10.552402 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:10.543379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:45247 (local address 127.15.77.67:39163)
0620 18:44:10.543561 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:10.543598 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:10.544080 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:10.544283 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:10.544286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:10.544851 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:10.545049 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:10.545054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.545707 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.545707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:10.547111 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:10.547114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.547516 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.547517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:10.547625 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:10.548059 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:10.548067 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:10.549508 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:10.551101 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:10.551107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:10.551110 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:10.551379 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:10.551549 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:10.551549 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:10.551550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:10.551658 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:10.551874 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:10.551878 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:10.551990 (+ 112us) client_negotiation.cc:769] Sending connection context
0620 18:44:10.552078 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 18:44:10.552194 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260620 18:44:10.552888 19185 heartbeater.cc:347] Connected to a master server at 127.15.77.126:45247
I20260620 18:44:10.553005 19185 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:10.553193 19185 heartbeater.cc:511] Master 127.15.77.126:45247 requested a full tablet report, sending...
I20260620 18:44:10.553619 18726 ts_manager.cc:194] Registered new tserver with Master: deef129257544d439a11352d60f5cedf (127.15.77.67:39239)
I20260620 18:44:10.554044 18726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:39163
I20260620 18:44:10.557928 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[18681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981050, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[18681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981050, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:10.580641 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:10.573113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58958 (local address 127.15.77.126:45247)
0620 18:44:10.573182 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:10.573185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:10.573300 (+ 115us) server_negotiation.cc:407] Connection header received
0620 18:44:10.573388 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:10.573389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:10.573404 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:10.573466 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:10.574065 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.574577 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.575359 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:10.575555 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:10.575639 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:10.577891 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:10.577893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:10.577893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:10.577900 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:10.579603 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:10.579944 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:10.579945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:10.579945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:10.579996 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:10.580165 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:10.580166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:10.580166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:10.580309 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:10.580387 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:10.580489 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 18:44:10.580549 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 18:44:10.580931 18726 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58958
I20260620 18:44:10.581434 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 18788
I20260620 18:44:10.586287 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 18922
I20260620 18:44:10.590806 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 19056
I20260620 18:44:10.594921 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 18694
2026-06-20T18:44:10Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1937 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[19197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[19197](info): set up 2 sockets
Jun 20 18:44:10 dist-test-slave-tvl9 krb5kdc[19197](info): commencing operation
krb5kdc: starting...
W20260620 18:44:11.631193 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:44:11 dist-test-slave-tvl9 krb5kdc[19197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:44:11.750705 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40463
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:11.832584 19210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:11.832862 19210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:11.832899 19210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:11.834100 19210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:11.834182 19210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:11.834205 19210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:11.834225 19210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:11.834254 19210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:11.835872 19210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:11.836170 19210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:11.836417 19210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:11.841414 19218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:11.841403 19216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:11.841403 19215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:11.841619 19210 server_base.cc:1034] running on GCE node
I20260620 18:44:11.841887 19210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:11.842192 19210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:11.843382 19210 hybrid_clock.cc:648] HybridClock initialized: now 1781981051843361 us; error 41 us; skew 500 ppm
Jun 20 18:44:11 dist-test-slave-tvl9 krb5kdc[19197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981051, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:11.845877 19210 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:44:11.846385 19210 webserver.cc:460] Webserver started at http://127.15.77.126:46607/ using document root <none> and password file <none>
I20260620 18:44:11.846650 19210 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:11.846724 19210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:11.846872 19210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:11.848003 19210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "3a069dea11d542be8bc9525c91a54347"
format_stamp: "Formatted at 2026-06-20 18:44:11 on dist-test-slave-tvl9"
server_key: "42b1797c67a147e3909fca41e1e150cf"
server_key_iv: "612d8561671adce9d85ee9134ff1aab6"
server_key_version: "encryptionkey@0"
I20260620 18:44:11.848376 19210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "3a069dea11d542be8bc9525c91a54347"
format_stamp: "Formatted at 2026-06-20 18:44:11 on dist-test-slave-tvl9"
server_key: "42b1797c67a147e3909fca41e1e150cf"
server_key_iv: "612d8561671adce9d85ee9134ff1aab6"
server_key_version: "encryptionkey@0"
I20260620 18:44:11.849731 19210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:44:11.850471 19225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:11.850641 19210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:11.850750 19210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "3a069dea11d542be8bc9525c91a54347"
format_stamp: "Formatted at 2026-06-20 18:44:11 on dist-test-slave-tvl9"
server_key: "42b1797c67a147e3909fca41e1e150cf"
server_key_iv: "612d8561671adce9d85ee9134ff1aab6"
server_key_version: "encryptionkey@0"
I20260620 18:44:11.850823 19210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:11.863044 19210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:11.865092 19210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:11.865278 19210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:11.869520 19210 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:40463
I20260620 18:44:11.869562 19277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:40463 every 8 connection(s)
I20260620 18:44:11.870080 19210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:11.870848 19278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:11.873165 19278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347: Bootstrap starting.
I20260620 18:44:11.873746 19278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:11.873989 19278 log.cc:824] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:11.874702 19278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347: No bootstrap required, opened a new log
I20260620 18:44:11.875705 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 19210
I20260620 18:44:11.875818 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:11.876003 15669 external_mini_cluster.cc:1428] Setting key 689b53564d8b6dc9bab5e06bcbcb7ae5
I20260620 18:44:11.877439 19278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a069dea11d542be8bc9525c91a54347" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40463 } }
I20260620 18:44:11.877586 19278 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:11.877609 19278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a069dea11d542be8bc9525c91a54347, State: Initialized, Role: FOLLOWER
I20260620 18:44:11.877751 19278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a069dea11d542be8bc9525c91a54347" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40463 } }
I20260620 18:44:11.877854 19278 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:11.877898 19278 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:11.877948 19278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:11.878510 19278 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a069dea11d542be8bc9525c91a54347" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40463 } }
I20260620 18:44:11.878633 19278 leader_election.cc:302] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a069dea11d542be8bc9525c91a54347; no voters:
I20260620 18:44:11.878816 19278 leader_election.cc:288] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:11.878861 19283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:11.879011 19283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [term 1 LEADER]: Becoming Leader. State: Replica: 3a069dea11d542be8bc9525c91a54347, State: Running, Role: LEADER
I20260620 18:44:11.879096 19278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:11.879156 19283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a069dea11d542be8bc9525c91a54347" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40463 } }
I20260620 18:44:11.879611 19284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a069dea11d542be8bc9525c91a54347" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a069dea11d542be8bc9525c91a54347" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40463 } } }
I20260620 18:44:11.879699 19284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:11.879927 19285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a069dea11d542be8bc9525c91a54347. Latest consensus state: current_term: 1 leader_uuid: "3a069dea11d542be8bc9525c91a54347" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a069dea11d542be8bc9525c91a54347" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40463 } } }
I20260620 18:44:11.880069 19285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:11.880224 19290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:11.880797 19290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:11.882490 19290 catalog_manager.cc:1347] Generated new cluster ID: 0b12a5e03c4442fbb59816a8465fb28a
I20260620 18:44:11.882548 19290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:44:11 dist-test-slave-tvl9 krb5kdc[19197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:11.887663 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:11.876591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.15.77.126:40463)
0620 18:44:11.876796 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:11.876804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:11.876835 (+ 31us) server_negotiation.cc:407] Connection header received
0620 18:44:11.877439 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:11.877448 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:11.877539 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:11.877745 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:11.878553 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:11.879933 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:11.881051 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:11.881666 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:11.881807 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:11.884307 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:11.884311 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:11.884312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:11.884324 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:11.886067 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:11.886452 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:11.886455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:11.886456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:11.886566 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:11.886786 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:11.886787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:11.886787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:11.887028 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:11.887221 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:11.887287 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 18:44:11.887447 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260620 18:44:11.897341 19290 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:11.897897 19290 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:11.904327 19290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a069dea11d542be8bc9525c91a54347: Generated new TSK 0
I20260620 18:44:11.904542 19290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:44:11.937861 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40463
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:12.021039 19304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:12.021262 19304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:12.021286 19304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:12.022554 19304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:12.022598 19304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:12.022625 19304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:44:12.024197 19304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:12.024471 19304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:12.024720 19304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:12.029951 19312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:12.030133 19309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:12.029958 19310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:12.030040 19304 server_base.cc:1034] running on GCE node
I20260620 18:44:12.030411 19304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:12.030694 19304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:12.031888 19304 hybrid_clock.cc:648] HybridClock initialized: now 1781981052031863 us; error 38 us; skew 500 ppm
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981052, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:12.034422 19304 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:44:12.034873 19304 webserver.cc:460] Webserver started at http://127.15.77.65:34405/ using document root <none> and password file <none>
I20260620 18:44:12.035115 19304 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:12.035184 19304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:12.035394 19304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:12.036512 19304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "2cd2705671314604b879fe9fc9387183"
format_stamp: "Formatted at 2026-06-20 18:44:12 on dist-test-slave-tvl9"
server_key: "384dde6428b04962c8fe301dcbcc7f95"
server_key_iv: "b56032698a169e25bd2108a90f3c14b8"
server_key_version: "encryptionkey@0"
I20260620 18:44:12.036880 19304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "2cd2705671314604b879fe9fc9387183"
format_stamp: "Formatted at 2026-06-20 18:44:12 on dist-test-slave-tvl9"
server_key: "384dde6428b04962c8fe301dcbcc7f95"
server_key_iv: "b56032698a169e25bd2108a90f3c14b8"
server_key_version: "encryptionkey@0"
I20260620 18:44:12.038239 19304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:12.038965 19319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:12.039145 19304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:12.039250 19304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "2cd2705671314604b879fe9fc9387183"
format_stamp: "Formatted at 2026-06-20 18:44:12 on dist-test-slave-tvl9"
server_key: "384dde6428b04962c8fe301dcbcc7f95"
server_key_iv: "b56032698a169e25bd2108a90f3c14b8"
server_key_version: "encryptionkey@0"
I20260620 18:44:12.039345 19304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:12.051970 19304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:12.053987 19304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:12.054145 19304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:12.054412 19304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:12.054781 19304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:12.054837 19304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:12.054872 19304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:12.054899 19304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:12.060748 19304 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:34133
I20260620 18:44:12.060824 19432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:34133 every 8 connection(s)
I20260620 18:44:12.061159 19304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:44:12.062842 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 19304
I20260620 18:44:12.062924 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:44:12.063114 15669 external_mini_cluster.cc:1428] Setting key 1267f44e029a6348e2d41a37e1e655bf
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981052, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:44:12.072010 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:12.061905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:35419 (local address 127.15.77.126:40463)
0620 18:44:12.061971 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:12.061973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:12.062201 (+ 228us) server_negotiation.cc:407] Connection header received
0620 18:44:12.063258 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:12.063259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:12.063277 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:12.063372 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:12.064178 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.064685 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.065701 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.065976 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.066044 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:12.068447 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:12.068449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:12.068450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:12.068460 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:12.070335 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:12.070786 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:12.070787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:12.070788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:12.070852 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:12.071159 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:12.071160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:12.071160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:12.071335 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:12.071412 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:12.071779 (+ 367us) server_negotiation.cc:299] Negotiation successful
0620 18:44:12.071893 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 18:44:12.072206 19435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:12.061891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40463 (local address 127.15.77.65:35419)
0620 18:44:12.062078 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:12.062115 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:12.063131 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:12.063470 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:12.063475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:12.063837 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:12.064065 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:12.064070 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.064781 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.064782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:12.065597 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:12.065599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.066065 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.066066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:12.066193 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:12.066784 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:12.066793 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:12.068336 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:12.070427 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:12.070434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:12.070436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:12.070702 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:12.070942 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:12.070943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:12.070943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:12.071070 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:12.071444 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:12.071452 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:12.071622 (+ 170us) client_negotiation.cc:769] Sending connection context
0620 18:44:12.071739 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 18:44:12.071925 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260620 18:44:12.072826 19433 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40463
I20260620 18:44:12.072978 19433 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:12.073170 19433 heartbeater.cc:511] Master 127.15.77.126:40463 requested a full tablet report, sending...
I20260620 18:44:12.073746 19242 ts_manager.cc:194] Registered new tserver with Master: 2cd2705671314604b879fe9fc9387183 (127.15.77.65:34133)
I20260620 18:44:12.074361 19242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:35419
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:44:12.089326 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40463
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:12.171094 19438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:12.171308 19438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:12.171407 19438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:12.172645 19438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:12.172688 19438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:12.172745 19438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:44:12.174261 19438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:12.174511 19438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:12.174751 19438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:12.179853 19444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:12.179921 19443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:12.179839 19446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:12.180056 19438 server_base.cc:1034] running on GCE node
I20260620 18:44:12.180294 19438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:12.180493 19438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:12.181635 19438 hybrid_clock.cc:648] HybridClock initialized: now 1781981052181623 us; error 27 us; skew 500 ppm
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981052, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:12.184288 19438 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:44:12.184768 19438 webserver.cc:460] Webserver started at http://127.15.77.66:37009/ using document root <none> and password file <none>
I20260620 18:44:12.184996 19438 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:12.185042 19438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:12.185184 19438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:12.186214 19438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "8e986459aab84970833adac38ddaec19"
format_stamp: "Formatted at 2026-06-20 18:44:12 on dist-test-slave-tvl9"
server_key: "48cafff730f492c7eeaef029919e1c34"
server_key_iv: "7dea1577a1b812e4775fea55203d6e4f"
server_key_version: "encryptionkey@0"
I20260620 18:44:12.186571 19438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "8e986459aab84970833adac38ddaec19"
format_stamp: "Formatted at 2026-06-20 18:44:12 on dist-test-slave-tvl9"
server_key: "48cafff730f492c7eeaef029919e1c34"
server_key_iv: "7dea1577a1b812e4775fea55203d6e4f"
server_key_version: "encryptionkey@0"
I20260620 18:44:12.187973 19438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:12.188681 19453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:12.188846 19438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:12.188951 19438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "8e986459aab84970833adac38ddaec19"
format_stamp: "Formatted at 2026-06-20 18:44:12 on dist-test-slave-tvl9"
server_key: "48cafff730f492c7eeaef029919e1c34"
server_key_iv: "7dea1577a1b812e4775fea55203d6e4f"
server_key_version: "encryptionkey@0"
I20260620 18:44:12.189029 19438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:12.203711 19438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:12.206039 19438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:12.206200 19438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:12.206435 19438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:12.206738 19438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:12.206791 19438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:12.206836 19438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:12.206866 19438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:12.212832 19438 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:45805
I20260620 18:44:12.212882 19566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:45805 every 8 connection(s)
I20260620 18:44:12.213212 19438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:44:12.213922 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 19438
I20260620 18:44:12.214052 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:44:12.214241 15669 external_mini_cluster.cc:1428] Setting key 62e0d5dd1adeb8edc484da03bbb4361e
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981052, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
I20260620 18:44:12.224299 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:12.214009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:42715 (local address 127.15.77.126:40463)
0620 18:44:12.214072 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:12.214075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:12.214307 (+ 232us) server_negotiation.cc:407] Connection header received
0620 18:44:12.215599 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:12.215600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:12.215614 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:12.215659 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:12.216409 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.216923 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.217948 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.218264 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.218336 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:12.221095 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:12.221097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:12.221097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:12.221107 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:12.222764 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:12.223197 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:12.223198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:12.223199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:12.223264 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:12.223580 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:12.223581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:12.223581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:12.223735 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:12.223808 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:12.224106 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 18:44:12.224203 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 18:44:12.224498 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:12.213969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40463 (local address 127.15.77.66:42715)
0620 18:44:12.214160 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:12.214205 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:12.215493 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:12.215738 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:12.215745 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:12.216013 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:12.216267 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:12.216273 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.217024 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.217025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:12.217852 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:12.217854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.218360 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.218361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:12.218517 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:12.219112 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:12.219120 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:12.220995 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:12.222862 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:12.222868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:12.222871 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:12.223113 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:12.223385 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:12.223386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:12.223386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:12.223500 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:12.223809 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:12.223814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:12.223940 (+ 126us) client_negotiation.cc:769] Sending connection context
0620 18:44:12.224078 (+ 138us) client_negotiation.cc:240] Negotiation successful
0620 18:44:12.224256 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260620 18:44:12.225131 19567 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40463
I20260620 18:44:12.225257 19567 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:12.225481 19567 heartbeater.cc:511] Master 127.15.77.126:40463 requested a full tablet report, sending...
I20260620 18:44:12.225984 19242 ts_manager.cc:194] Registered new tserver with Master: 8e986459aab84970833adac38ddaec19 (127.15.77.66:45805)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
I20260620 18:44:12.226605 19242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:42715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:44:12.241887 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40463
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:12.323494 19572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:12.323755 19572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:12.323796 19572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:12.324996 19572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:12.325067 19572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:12.325112 19572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:44:12.326671 19572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:12.326952 19572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:12.327200 19572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:12.332181 19580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:12.332190 19577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:12.332181 19578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:12.332535 19572 server_base.cc:1034] running on GCE node
I20260620 18:44:12.332862 19572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:12.333123 19572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:12.334285 19572 hybrid_clock.cc:648] HybridClock initialized: now 1781981052334267 us; error 33 us; skew 500 ppm
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981052, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:12.336876 19572 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:44:12.337352 19572 webserver.cc:460] Webserver started at http://127.15.77.67:41261/ using document root <none> and password file <none>
I20260620 18:44:12.337608 19572 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:12.337680 19572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:12.337879 19572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:12.338990 19572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "d42273d187644a87b6da73d19481bda3"
format_stamp: "Formatted at 2026-06-20 18:44:12 on dist-test-slave-tvl9"
server_key: "1bec721f6e617c37252975e3daa77c8b"
server_key_iv: "472cd3c7354a17e62cf524f4bd157cb0"
server_key_version: "encryptionkey@0"
I20260620 18:44:12.339418 19572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "d42273d187644a87b6da73d19481bda3"
format_stamp: "Formatted at 2026-06-20 18:44:12 on dist-test-slave-tvl9"
server_key: "1bec721f6e617c37252975e3daa77c8b"
server_key_iv: "472cd3c7354a17e62cf524f4bd157cb0"
server_key_version: "encryptionkey@0"
I20260620 18:44:12.340842 19572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:12.341601 19587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:12.341773 19572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:12.341876 19572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "d42273d187644a87b6da73d19481bda3"
format_stamp: "Formatted at 2026-06-20 18:44:12 on dist-test-slave-tvl9"
server_key: "1bec721f6e617c37252975e3daa77c8b"
server_key_iv: "472cd3c7354a17e62cf524f4bd157cb0"
server_key_version: "encryptionkey@0"
I20260620 18:44:12.341960 19572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:12.353211 19572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:12.355286 19572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:12.355510 19572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:12.355775 19572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:12.356109 19572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:12.356142 19572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:12.356186 19572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:12.356215 19572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:12.362094 19572 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:37651
I20260620 18:44:12.362135 19700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:37651 every 8 connection(s)
I20260620 18:44:12.362464 19572 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:44:12.366849 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 19572
I20260620 18:44:12.366945 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:44:12.367170 15669 external_mini_cluster.cc:1428] Setting key 31c65835444b561d0f035fc9f08d56a1
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981052, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:12.372037 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:12.363201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:33353 (local address 127.15.77.126:40463)
0620 18:44:12.363252 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:12.363255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:12.363558 (+ 303us) server_negotiation.cc:407] Connection header received
0620 18:44:12.364468 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:12.364469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:12.364480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:12.364534 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:12.365124 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.365613 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.366735 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.367035 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.367119 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:12.369457 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:12.369458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:12.369459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:12.369466 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:12.370649 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:12.371017 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:12.371018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:12.371018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:12.371070 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:12.371374 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:12.371376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:12.371376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:12.371523 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:12.371593 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:12.371883 (+ 290us) server_negotiation.cc:299] Negotiation successful
0620 18:44:12.371968 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:44:12.372128 19703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:12.363208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40463 (local address 127.15.77.67:33353)
0620 18:44:12.363420 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:12.363459 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:12.364380 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:12.364590 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:12.364593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:12.364848 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:12.365053 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:12.365057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.365718 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.365719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:12.366651 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:12.366654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.367128 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.367129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:12.367278 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:12.367911 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:12.367921 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:12.369373 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:12.370719 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:12.370725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:12.370728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:12.370949 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:12.371130 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:12.371130 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:12.371131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:12.371237 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:12.371591 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:12.371596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:12.371735 (+ 139us) client_negotiation.cc:769] Sending connection context
0620 18:44:12.371827 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 18:44:12.371954 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":109,"threads_started":1}
I20260620 18:44:12.372653 19701 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40463
I20260620 18:44:12.372743 19701 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:12.372934 19701 heartbeater.cc:511] Master 127.15.77.126:40463 requested a full tablet report, sending...
I20260620 18:44:12.373365 19242 ts_manager.cc:194] Registered new tserver with Master: d42273d187644a87b6da73d19481bda3 (127.15.77.67:37651)
I20260620 18:44:12.373765 19242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:33353
I20260620 18:44:12.379197 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:44:12.402757 19708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:12.394735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47154 (local address 127.15.77.65:34133)
0620 18:44:12.394908 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:12.394914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:12.394924 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:12.394951 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:12.394953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:12.394968 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:12.395035 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:12.395679 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.396382 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.397128 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.397454 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.397508 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:12.399891 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:12.399901 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:12.399901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:12.399908 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:12.401608 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:12.401940 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:12.401943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:12.401944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:12.402006 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:12.402178 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:12.402179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:12.402179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:12.402314 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:12.402420 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:12.402520 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 18:44:12.402583 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:12.410756 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:12.404085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.15.77.126:40463)
0620 18:44:12.404134 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:12.404137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:12.404214 (+ 77us) server_negotiation.cc:407] Connection header received
0620 18:44:12.404292 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:12.404292 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:12.404309 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:12.404375 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:12.404914 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.405405 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.406091 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:12.406280 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:12.406362 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:12.408623 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:12.408624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:12.408624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:12.408631 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:12.409815 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:12.410130 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:12.410131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:12.410132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:12.410179 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:12.410339 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:12.410340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:12.410340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:12.410477 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:12.410549 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:12.410633 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 18:44:12.410689 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 18:44:12.411034 19242 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59382
I20260620 18:44:12.411394 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 19304
I20260620 18:44:12.416112 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 19438
I20260620 18:44:12.420740 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 19572
I20260620 18:44:12.425417 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 19210
2026-06-20T18:44:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1829 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19713](info): set up 2 sockets
Jun 20 18:44:12 dist-test-slave-tvl9 krb5kdc[19713](info): commencing operation
krb5kdc: starting...
W20260620 18:44:13.463251 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260620 18:44:13.700160 15669 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.121s user 0.002s sys 0.000s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/test-user.keytab.
W20260620 18:44:14.453327 15669 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.753s user 0.002s sys 0.000s
Jun 20 18:44:14 dist-test-slave-tvl9 krb5kdc[19713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:44:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
W20260620 18:44:15.653165 15669 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.77.126: real 1.178s user 0.000s sys 0.003s
I20260620 18:44:15.653550 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:42925
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:45685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:42925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:15.737427 19726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:15.737656 19726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:15.737681 19726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:15.739681 19726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:15.739748 19726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:15.739773 19726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:15.739806 19726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:15.739826 19726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:15.741930 19726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:42925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:42925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:15.742168 19726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:15.742352 19726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:15.747540 19734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:15.747577 19726 server_base.cc:1034] running on GCE node
W20260620 18:44:15.747565 19732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:15.747556 19731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:15.748193 19726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:15.748533 19726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:15.749715 19726 hybrid_clock.cc:648] HybridClock initialized: now 1781981055749701 us; error 31 us; skew 500 ppm
Jun 20 18:44:15 dist-test-slave-tvl9 krb5kdc[19713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981055, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:15.757416 19726 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:44:15.757833 19726 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:15.757913 19726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:15.758078 19726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:15.759265 19726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "b68f7a17e3c7491ba3971a71ed76f0ac"
format_stamp: "Formatted at 2026-06-20 18:44:15 on dist-test-slave-tvl9"
server_key: "0d54ae590db2dd09a7f4424d01c78841"
server_key_iv: "7932dd10815347a457a1debf0b95e4e6"
server_key_version: "encryptionkey@0"
I20260620 18:44:15.759730 19726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "b68f7a17e3c7491ba3971a71ed76f0ac"
format_stamp: "Formatted at 2026-06-20 18:44:15 on dist-test-slave-tvl9"
server_key: "0d54ae590db2dd09a7f4424d01c78841"
server_key_iv: "7932dd10815347a457a1debf0b95e4e6"
server_key_version: "encryptionkey@0"
I20260620 18:44:15.761307 19726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:44:15.762094 19740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:15.762308 19726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:15.762423 19726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "b68f7a17e3c7491ba3971a71ed76f0ac"
format_stamp: "Formatted at 2026-06-20 18:44:15 on dist-test-slave-tvl9"
server_key: "0d54ae590db2dd09a7f4424d01c78841"
server_key_iv: "7932dd10815347a457a1debf0b95e4e6"
server_key_version: "encryptionkey@0"
I20260620 18:44:15.762508 19726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:15.774924 19726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:15.777064 19726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:15.777261 19726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:15.781343 19726 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:42925
I20260620 18:44:15.781370 19792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:42925 every 8 connection(s)
I20260620 18:44:15.781695 19726 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:15.782721 19793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:15.785439 19793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac: Bootstrap starting.
I20260620 18:44:15.786120 19793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:15.786440 19793 log.cc:824] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:15.787227 19793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac: No bootstrap required, opened a new log
I20260620 18:44:15.788839 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 19726
I20260620 18:44:15.788975 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:15.788968 19793 raft_consensus.cc:348] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68f7a17e3c7491ba3971a71ed76f0ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 42925 } }
I20260620 18:44:15.789157 19793 raft_consensus.cc:374] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:15.789193 15669 external_mini_cluster.cc:1428] Setting key 277e84732798f7238dde68672beda26b
I20260620 18:44:15.789203 19793 raft_consensus.cc:729] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68f7a17e3c7491ba3971a71ed76f0ac, State: Initialized, Role: FOLLOWER
I20260620 18:44:15.789381 19793 consensus_queue.cc:260] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68f7a17e3c7491ba3971a71ed76f0ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 42925 } }
I20260620 18:44:15.789486 19793 raft_consensus.cc:388] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:15.789529 19793 raft_consensus.cc:482] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:15.789583 19793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [term 0 FOLLOWER]: Advancing to term 1
Jun 20 18:44:15 dist-test-slave-tvl9 krb5kdc[19713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:15.799049 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:15.789839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.15.77.126:42925)
0620 18:44:15.790072 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:15.790081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:15.790108 (+ 27us) server_negotiation.cc:407] Connection header received
0620 18:44:15.790221 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:15.790226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:15.790278 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:15.790424 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:15.791065 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:15.792069 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:15.792881 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:15.793320 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:15.793444 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:15.795926 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:15.795931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:15.795931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:15.795944 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:15.797579 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:15.797992 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:15.797995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:15.797995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:15.798093 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:15.798274 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:15.798275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:15.798275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:15.798530 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:15.798648 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:15.798766 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 18:44:15.798875 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":95,"threads_started":1}
I20260620 18:44:15.848253 19793 raft_consensus.cc:504] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68f7a17e3c7491ba3971a71ed76f0ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 42925 } }
I20260620 18:44:15.848484 19793 leader_election.cc:302] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b68f7a17e3c7491ba3971a71ed76f0ac; no voters:
I20260620 18:44:15.848726 19793 leader_election.cc:288] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:15.848819 19798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:15.849011 19798 raft_consensus.cc:686] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [term 1 LEADER]: Becoming Leader. State: Replica: b68f7a17e3c7491ba3971a71ed76f0ac, State: Running, Role: LEADER
I20260620 18:44:15.849071 19793 sys_catalog.cc:564] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:15.849141 19798 consensus_queue.cc:237] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68f7a17e3c7491ba3971a71ed76f0ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 42925 } }
I20260620 18:44:15.849534 19799 sys_catalog.cc:455] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b68f7a17e3c7491ba3971a71ed76f0ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68f7a17e3c7491ba3971a71ed76f0ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 42925 } } }
I20260620 18:44:15.849642 19799 sys_catalog.cc:458] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:15.849924 19800 sys_catalog.cc:455] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader b68f7a17e3c7491ba3971a71ed76f0ac. Latest consensus state: current_term: 1 leader_uuid: "b68f7a17e3c7491ba3971a71ed76f0ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b68f7a17e3c7491ba3971a71ed76f0ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 42925 } } }
I20260620 18:44:15.850036 19800 sys_catalog.cc:458] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:15.850217 19806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:15.851191 19806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:15.853174 19806 catalog_manager.cc:1347] Generated new cluster ID: 14c3efcbbb654216b21bbbfa5950d85e
I20260620 18:44:15.853231 19806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:44:15.858886 19806 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:15.859582 19806 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:15.866768 19806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac: Generated new TSK 0
I20260620 18:44:15.866993 19806 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
W20260620 18:44:17.060194 15669 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.77.65: real 1.186s user 0.001s sys 0.002s
I20260620 18:44:17.060591 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:42925
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:45685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:17.146116 19819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:17.146348 19819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:17.146377 19819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:17.147619 19819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:17.147670 19819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:17.147698 19819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:44:17.149343 19819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:42925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:17.149614 19819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:17.149884 19819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:17.155470 19825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:17.155611 19824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:17.155642 19819 server_base.cc:1034] running on GCE node
W20260620 18:44:17.155470 19827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:17.155979 19819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:17.156204 19819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:17.157371 19819 hybrid_clock.cc:648] HybridClock initialized: now 1781981057157355 us; error 43 us; skew 500 ppm
Jun 20 18:44:17 dist-test-slave-tvl9 krb5kdc[19713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981057, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:17.160117 19819 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:44:17.160419 19819 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:17.160466 19819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:17.160558 19819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:17.161594 19819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "37bf17d7fc54402ca1fa16f816b2b5e2"
format_stamp: "Formatted at 2026-06-20 18:44:17 on dist-test-slave-tvl9"
server_key: "a3f0b69367f8f53a5f16ea1860d1dace"
server_key_iv: "5b7c72c7d6d951ac697eaf1ab6b38858"
server_key_version: "encryptionkey@0"
I20260620 18:44:17.161957 19819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "37bf17d7fc54402ca1fa16f816b2b5e2"
format_stamp: "Formatted at 2026-06-20 18:44:17 on dist-test-slave-tvl9"
server_key: "a3f0b69367f8f53a5f16ea1860d1dace"
server_key_iv: "5b7c72c7d6d951ac697eaf1ab6b38858"
server_key_version: "encryptionkey@0"
I20260620 18:44:17.163354 19819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:44:17.164198 19833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:17.164422 19819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:17.164543 19819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "37bf17d7fc54402ca1fa16f816b2b5e2"
format_stamp: "Formatted at 2026-06-20 18:44:17 on dist-test-slave-tvl9"
server_key: "a3f0b69367f8f53a5f16ea1860d1dace"
server_key_iv: "5b7c72c7d6d951ac697eaf1ab6b38858"
server_key_version: "encryptionkey@0"
I20260620 18:44:17.164618 19819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:17.191640 19819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:17.194207 19819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:17.194381 19819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:17.194661 19819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:17.195063 19819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:17.195123 19819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:17.195158 19819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:17.195185 19819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:17.201524 19819 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:35977
I20260620 18:44:17.201572 19946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:35977 every 8 connection(s)
I20260620 18:44:17.201851 19819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:44:17.206570 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 19819
I20260620 18:44:17.206707 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:44:17.206897 15669 external_mini_cluster.cc:1428] Setting key 89da9cb94dd2df10753cc0324afbf0e4
Jun 20 18:44:17 dist-test-slave-tvl9 krb5kdc[19713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981057, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:17.212841 19950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:17.202631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:42925 (local address 127.15.77.65:34259)
0620 18:44:17.202871 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:17.202936 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:17.203497 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:17.203783 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:17.203788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:17.204069 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:17.204580 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:17.204587 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:17.205353 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:17.205355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:17.206188 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:17.206190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:17.206639 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:17.206640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:17.206830 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:17.207468 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:17.207477 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:17.209275 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:17.211462 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:17.211470 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:17.211472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:17.211703 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:17.211915 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:17.211916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:17.211916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:17.212017 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:17.212333 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:17.212338 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:17.212456 (+ 118us) client_negotiation.cc:769] Sending connection context
0620 18:44:17.212539 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 18:44:17.212668 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":82,"threads_started":1}
I20260620 18:44:17.212853 19949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:17.202646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:34259 (local address 127.15.77.126:42925)
0620 18:44:17.202861 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:17.202864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:17.203040 (+ 176us) server_negotiation.cc:407] Connection header received
0620 18:44:17.203619 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:17.203621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:17.203634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:17.203696 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:17.204698 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:17.205251 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:17.206285 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:17.206531 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:17.206624 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:17.209378 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:17.209380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:17.209380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:17.209387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:17.211368 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:17.211764 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:17.211766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:17.211767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:17.211843 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:17.212100 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:17.212101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:17.212101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:17.212264 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:17.212342 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:17.212595 (+ 253us) server_negotiation.cc:299] Negotiation successful
0620 18:44:17.212700 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
I20260620 18:44:17.213400 19947 heartbeater.cc:347] Connected to a master server at 127.15.77.126:42925
I20260620 18:44:17.213522 19947 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:17.213676 19947 heartbeater.cc:511] Master 127.15.77.126:42925 requested a full tablet report, sending...
I20260620 18:44:17.214316 19757 ts_manager.cc:194] Registered new tserver with Master: 37bf17d7fc54402ca1fa16f816b2b5e2 (127.15.77.65:35977)
I20260620 18:44:17.215286 19757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:34259
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
W20260620 18:44:17.963907 15669 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.77.66: real 0.756s user 0.001s sys 0.002s
I20260620 18:44:17.964273 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:42925
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:45685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:18.048197 19953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:18.048463 19953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:18.048508 19953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:18.049865 19953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:18.049955 19953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:18.050051 19953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:44:18.051760 19953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:42925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:18.052052 19953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:18.052304 19953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:18.057581 19961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:18.057605 19959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:18.057659 19953 server_base.cc:1034] running on GCE node
W20260620 18:44:18.057641 19958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:18.058091 19953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:18.058367 19953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:18.059537 19953 hybrid_clock.cc:648] HybridClock initialized: now 1781981058059513 us; error 41 us; skew 500 ppm
Jun 20 18:44:18 dist-test-slave-tvl9 krb5kdc[19713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981058, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:18.062026 19953 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:44:18.062402 19953 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:18.062453 19953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:18.062543 19953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:18.063848 19953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "90572ad288ce47b9a702cef4292ab94a"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "2b806986495192d09e67ea1600b00090"
server_key_iv: "69b5fa9611c27d4bc5a5e61eadf3e5d9"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.064253 19953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "90572ad288ce47b9a702cef4292ab94a"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "2b806986495192d09e67ea1600b00090"
server_key_iv: "69b5fa9611c27d4bc5a5e61eadf3e5d9"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.065681 19953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:44:18.066486 19967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.066679 19953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:18.066789 19953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "90572ad288ce47b9a702cef4292ab94a"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "2b806986495192d09e67ea1600b00090"
server_key_iv: "69b5fa9611c27d4bc5a5e61eadf3e5d9"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.066870 19953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:18.079692 19953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:18.081907 19953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:18.082029 19953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:18.082240 19953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:18.082613 19953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:18.082650 19953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.082674 19953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:18.082724 19953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.088807 19953 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:44955
I20260620 18:44:18.088888 20080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:44955 every 8 connection(s)
I20260620 18:44:18.089178 19953 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:44:18.089496 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 19953
I20260620 18:44:18.089607 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:44:18.089807 15669 external_mini_cluster.cc:1428] Setting key 01aa43ac637bb8fab44dc03c2a9a2aba
Jun 20 18:44:18 dist-test-slave-tvl9 krb5kdc[19713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981058, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
I20260620 18:44:18.099944 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.089984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:48607 (local address 127.15.77.126:42925)
0620 18:44:18.090183 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.090185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.090318 (+ 133us) server_negotiation.cc:407] Connection header received
0620 18:44:18.090920 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.090920 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.090936 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.091019 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.092204 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.092748 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.093798 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.094150 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.094234 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.096770 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.096772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.096772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:18.096781 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.098492 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:18.098872 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:18.098873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:18.098874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:18.098932 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:18.099233 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:18.099234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:18.099235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:18.099463 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.099522 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.099794 (+ 272us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.099867 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
I20260620 18:44:18.100148 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.090001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:42925 (local address 127.15.77.66:48607)
0620 18:44:18.090186 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.090230 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.090809 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.091075 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.091079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.091414 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:18.092095 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.092103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.092850 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.092851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.093700 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.093702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.094240 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.094241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.094350 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.094937 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:18.094946 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:18.096668 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:18.098584 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:18.098591 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:18.098593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:18.098803 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:18.099017 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:18.099018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:18.099018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:18.099143 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:18.099532 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:18.099538 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:18.099674 (+ 136us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.099764 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.099890 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260620 18:44:18.100661 20081 heartbeater.cc:347] Connected to a master server at 127.15.77.126:42925
I20260620 18:44:18.100782 20081 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
I20260620 18:44:18.100973 20081 heartbeater.cc:511] Master 127.15.77.126:42925 requested a full tablet report, sending...
I20260620 18:44:18.101389 19757 ts_manager.cc:194] Registered new tserver with Master: 90572ad288ce47b9a702cef4292ab94a (127.15.77.66:44955)
I20260620 18:44:18.101794 19757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:48607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:44:18.115353 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:42925
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:45685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:18.198403 20087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:18.198616 20087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:18.198639 20087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:18.199859 20087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:18.199903 20087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:18.199929 20087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:44:18.201560 20087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:42925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:18.201814 20087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:18.202077 20087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:18.207154 20093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:18.207209 20092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:18.207387 20095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:18.207404 20087 server_base.cc:1034] running on GCE node
I20260620 18:44:18.207692 20087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:18.207942 20087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:18.209112 20087 hybrid_clock.cc:648] HybridClock initialized: now 1781981058209107 us; error 37 us; skew 500 ppm
Jun 20 18:44:18 dist-test-slave-tvl9 krb5kdc[19713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981058, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:18.211840 20087 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:44:18.212214 20087 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:18.212288 20087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:18.212440 20087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:18.213495 20087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "8f8e480bc1154abc9aaf562d79954e6d"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "42c91e582860b1237f172eada17391b2"
server_key_iv: "bd72f863c9d64d10ad0900153fe7bc61"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.213860 20087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "8f8e480bc1154abc9aaf562d79954e6d"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "42c91e582860b1237f172eada17391b2"
server_key_iv: "bd72f863c9d64d10ad0900153fe7bc61"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.215144 20087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:44:18.215996 20101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.216184 20087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:18.216285 19947 heartbeater.cc:503] Master 127.15.77.126:42925 was elected leader, sending a full tablet report...
I20260620 18:44:18.216286 20087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "8f8e480bc1154abc9aaf562d79954e6d"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "42c91e582860b1237f172eada17391b2"
server_key_iv: "bd72f863c9d64d10ad0900153fe7bc61"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.216363 20087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:18.230618 20087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:18.233165 20087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:18.233335 20087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:18.233590 20087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:18.233907 20087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:18.233961 20087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.234005 20087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:18.234030 20087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.240258 20087 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:40581
I20260620 18:44:18.240304 20214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:40581 every 8 connection(s)
I20260620 18:44:18.240615 20087 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
Jun 20 18:44:18 dist-test-slave-tvl9 krb5kdc[19713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981058, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:18.250273 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.241370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:44927 (local address 127.15.77.126:42925)
0620 18:44:18.241417 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.241419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.241667 (+ 248us) server_negotiation.cc:407] Connection header received
0620 18:44:18.242195 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.242196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.242209 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.242262 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.243256 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.243830 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.244870 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.245119 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.245209 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.247510 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.247511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.247512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:18.247519 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.248912 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:18.249317 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:18.249318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:18.249318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:18.249369 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:18.249688 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:18.249688 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:18.249689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:18.249799 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.249855 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.250112 (+ 257us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.250205 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:44:18.250365 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 20087
I20260620 18:44:18.250474 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:44:18.250584 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.241363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:42925 (local address 127.15.77.67:44927)
0620 18:44:18.241521 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.241559 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.242088 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.242346 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.242351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.242632 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:18.243148 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.243153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.243939 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.243941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.244761 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.244763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.245221 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.245222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.245354 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.245953 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:18.245962 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:18.247415 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:18.249009 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:18.249015 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:18.249017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:18.249249 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:18.249470 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:18.249471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:18.249472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:18.249613 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:18.249860 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:18.249864 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:18.250002 (+ 138us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.250091 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.250268 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260620 18:44:18.250684 15669 external_mini_cluster.cc:1428] Setting key 68e33472024a9b09553d04878b59bb98
I20260620 18:44:18.251183 20215 heartbeater.cc:347] Connected to a master server at 127.15.77.126:42925
I20260620 18:44:18.251293 20215 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:18.251482 20215 heartbeater.cc:511] Master 127.15.77.126:42925 requested a full tablet report, sending...
I20260620 18:44:18.251907 19756 ts_manager.cc:194] Registered new tserver with Master: 8f8e480bc1154abc9aaf562d79954e6d (127.15.77.67:40581)
I20260620 18:44:18.252260 19756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:44927
I20260620 18:44:18.262755 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:44:18.270342 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.264151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.15.77.126:42925)
0620 18:44:18.264217 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.264219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.264345 (+ 126us) server_negotiation.cc:407] Connection header received
0620 18:44:18.264389 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.264390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.264399 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.264450 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.265033 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.265541 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.266321 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.266500 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.266555 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.267813 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.267814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.267814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:18.267821 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.269211 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:18.269553 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:18.269554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:18.269554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:18.269604 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:18.269800 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:18.269801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:18.269801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:18.270030 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.270132 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.270213 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.270277 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 18:44:18.272061 19756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:44:18.273660 19756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:18.280732 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.276795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.15.77.66:44955)
0620 18:44:18.277005 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.277011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.277021 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:18.277062 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.277064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.277081 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.277155 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:18.277384 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.278086 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.279612 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.280204 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.280281 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.280354 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.280609 (+ 255us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.280648 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":90,"threads_started":1}
I20260620 18:44:18.280745 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.276330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:40581 (local address 127.0.0.1:51334)
0620 18:44:18.276540 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.276560 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.276627 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.277275 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.277276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.277277 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:18.277305 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.277306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.278484 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.278485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.279611 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.279613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.280486 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.280487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.280555 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.280558 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.280609 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.280644 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":2,"thread_start_us":98,"threads_started":1}
I20260620 18:44:18.280826 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.276405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.15.77.65:35977)
0620 18:44:18.276806 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.276811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.276821 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:18.276863 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.276864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.276877 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.276964 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:18.277152 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.278308 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.279522 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.280273 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.280354 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.280413 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.280694 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.280741 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":57,"threads_started":1}
I20260620 18:44:18.281107 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.276439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51334 (local address 127.15.77.67:40581)
0620 18:44:18.277107 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.277112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.277124 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:44:18.277183 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.277186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.277204 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.277280 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:18.277355 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.278383 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.279690 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.280384 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.280476 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.280528 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.280982 (+ 454us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.281029 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":67,"threads_started":1}
I20260620 18:44:18.280746 20231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.276542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:44955 (local address 127.0.0.1:33626)
0620 18:44:18.276673 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.276684 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.276734 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.277250 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.277251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.277253 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:18.277300 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.277302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.278194 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.278195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.279506 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.279509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.280382 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.280383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.280456 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.280475 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.280546 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.280644 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":63,"thread_start_us":68,"threads_started":1}
I20260620 18:44:18.281530 20228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.276290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:35977 (local address 127.0.0.1:53840)
0620 18:44:18.276508 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.276556 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.276627 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.276980 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.276983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.276985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:18.277064 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.277066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.278408 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.278409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.279419 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.279421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.280370 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.280371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.280456 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.280475 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.280546 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.280619 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260620 18:44:18.281963 20149 tablet_service.cc:1467] Processing CreateTablet for tablet af1ac1505e7f41c7be19effd5c31c83f (DEFAULT_TABLE table=test-table [id=74a692bd4d6443c6a46ea76d796ab90d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:18.282248 20015 tablet_service.cc:1467] Processing CreateTablet for tablet af1ac1505e7f41c7be19effd5c31c83f (DEFAULT_TABLE table=test-table [id=74a692bd4d6443c6a46ea76d796ab90d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:18.282245 19881 tablet_service.cc:1467] Processing CreateTablet for tablet af1ac1505e7f41c7be19effd5c31c83f (DEFAULT_TABLE table=test-table [id=74a692bd4d6443c6a46ea76d796ab90d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:18.282394 20149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1ac1505e7f41c7be19effd5c31c83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:18.282593 20015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1ac1505e7f41c7be19effd5c31c83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:18.282593 19881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af1ac1505e7f41c7be19effd5c31c83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:18.285223 20233 tablet_bootstrap.cc:492] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: Bootstrap starting.
I20260620 18:44:18.285871 20234 tablet_bootstrap.cc:492] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2: Bootstrap starting.
I20260620 18:44:18.285874 20233 tablet_bootstrap.cc:654] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:18.286659 20233 log.cc:824] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:18.286720 20234 tablet_bootstrap.cc:654] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:18.287024 20234 log.cc:824] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:18.287043 20235 tablet_bootstrap.cc:492] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: Bootstrap starting.
I20260620 18:44:18.287525 20233 tablet_bootstrap.cc:492] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: No bootstrap required, opened a new log
I20260620 18:44:18.287585 20233 ts_tablet_manager.cc:1397] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:44:18.287732 20235 tablet_bootstrap.cc:654] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:18.288208 20235 log.cc:824] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:18.288918 20235 tablet_bootstrap.cc:492] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: No bootstrap required, opened a new log
I20260620 18:44:18.289000 20235 ts_tablet_manager.cc:1397] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 18:44:18.289021 20234 tablet_bootstrap.cc:492] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2: No bootstrap required, opened a new log
I20260620 18:44:18.289080 20234 ts_tablet_manager.cc:1397] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 18:44:18.289393 20233 raft_consensus.cc:348] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35977 } } peers { permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 } } peers { permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 } }
I20260620 18:44:18.289566 20233 raft_consensus.cc:374] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:18.289605 20233 raft_consensus.cc:729] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f8e480bc1154abc9aaf562d79954e6d, State: Initialized, Role: FOLLOWER
I20260620 18:44:18.289724 20233 consensus_queue.cc:260] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35977 } } peers { permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 } } peers { permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 } }
I20260620 18:44:18.290007 20233 ts_tablet_manager.cc:1428] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 18:44:18.290766 20235 raft_consensus.cc:348] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35977 } } peers { permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 } } peers { permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 } }
I20260620 18:44:18.290912 20235 raft_consensus.cc:374] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:18.290958 20235 raft_consensus.cc:729] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90572ad288ce47b9a702cef4292ab94a, State: Initialized, Role: FOLLOWER
I20260620 18:44:18.291097 20235 consensus_queue.cc:260] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35977 } } peers { permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 } } peers { permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 } }
I20260620 18:44:18.291401 20215 heartbeater.cc:503] Master 127.15.77.126:42925 was elected leader, sending a full tablet report...
I20260620 18:44:18.291394 20081 heartbeater.cc:503] Master 127.15.77.126:42925 was elected leader, sending a full tablet report...
I20260620 18:44:18.291410 20235 ts_tablet_manager.cc:1428] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 18:44:18.291404 20234 raft_consensus.cc:348] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35977 } } peers { permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 } } peers { permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 } }
I20260620 18:44:18.291544 20234 raft_consensus.cc:374] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:18.291575 20234 raft_consensus.cc:729] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37bf17d7fc54402ca1fa16f816b2b5e2, State: Initialized, Role: FOLLOWER
I20260620 18:44:18.291852 20234 consensus_queue.cc:260] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35977 } } peers { permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 } } peers { permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 } }
I20260620 18:44:18.292202 20234 ts_tablet_manager.cc:1428] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:44:18.322482 20241 raft_consensus.cc:482] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:18.322613 20241 raft_consensus.cc:504] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35977 } } peers { permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 } } peers { permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 } }
I20260620 18:44:18.322932 20241 leader_election.cc:288] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90572ad288ce47b9a702cef4292ab94a (127.15.77.66:44955), 8f8e480bc1154abc9aaf562d79954e6d (127.15.77.67:40581)
I20260620 18:44:18.326293 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.323369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:51037 (local address 127.15.77.66:44955)
0620 18:44:18.323419 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.323422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.323490 (+ 68us) server_negotiation.cc:407] Connection header received
0620 18:44:18.323618 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.323619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.323654 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.323704 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:18.323895 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.324425 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.325542 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.325913 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.325971 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.325979 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.326167 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.326216 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:44:18.326308 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.323158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:44955 (local address 127.15.77.65:51037)
0620 18:44:18.323388 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.323403 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.323514 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.323721 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.323721 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.323723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:18.323793 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.323794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.324542 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.324543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.325437 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.325440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.326004 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.326005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.326087 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.326091 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.326140 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.326196 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":102,"spinlock_wait_cycles":12672,"thread_start_us":132,"threads_started":1}
I20260620 18:44:18.326328 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.323388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:43739 (local address 127.15.77.67:40581)
0620 18:44:18.323435 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.323437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.323498 (+ 61us) server_negotiation.cc:407] Connection header received
0620 18:44:18.323591 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.323592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.323604 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.323659 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:18.323910 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.324398 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.325539 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.325927 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.326020 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.326029 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.326226 (+ 197us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.326267 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:44:18.326337 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.323155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:40581 (local address 127.15.77.65:43739)
0620 18:44:18.323436 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.323445 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.323510 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.323697 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.323699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.323701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:18.323786 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.323788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.324488 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.324489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.325433 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:18.325436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.326051 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.326052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:18.326135 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.326140 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.326199 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.326238 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":19,"thread_start_us":122,"threads_started":1}
I20260620 18:44:18.326740 20035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1ac1505e7f41c7be19effd5c31c83f" candidate_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90572ad288ce47b9a702cef4292ab94a" is_pre_election: true
I20260620 18:44:18.326948 20035 raft_consensus.cc:2454] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37bf17d7fc54402ca1fa16f816b2b5e2 in term 0.
I20260620 18:44:18.326973 20169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1ac1505e7f41c7be19effd5c31c83f" candidate_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f8e480bc1154abc9aaf562d79954e6d" is_pre_election: true
I20260620 18:44:18.327142 20169 raft_consensus.cc:2454] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37bf17d7fc54402ca1fa16f816b2b5e2 in term 0.
I20260620 18:44:18.327226 19834 leader_election.cc:302] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37bf17d7fc54402ca1fa16f816b2b5e2, 90572ad288ce47b9a702cef4292ab94a; no voters:
I20260620 18:44:18.327386 20241 raft_consensus.cc:2781] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:18.327481 20241 raft_consensus.cc:482] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:18.327520 20241 raft_consensus.cc:3037] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:18.328243 20241 raft_consensus.cc:504] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35977 } } peers { permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 } } peers { permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 } }
I20260620 18:44:18.328404 20241 leader_election.cc:288] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [CANDIDATE]: Term 1 election: Requested vote from peers 90572ad288ce47b9a702cef4292ab94a (127.15.77.66:44955), 8f8e480bc1154abc9aaf562d79954e6d (127.15.77.67:40581)
I20260620 18:44:18.328552 20169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1ac1505e7f41c7be19effd5c31c83f" candidate_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f8e480bc1154abc9aaf562d79954e6d"
I20260620 18:44:18.328585 20035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af1ac1505e7f41c7be19effd5c31c83f" candidate_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90572ad288ce47b9a702cef4292ab94a"
I20260620 18:44:18.328660 20169 raft_consensus.cc:3037] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:18.328685 20035 raft_consensus.cc:3037] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:18.329412 20035 raft_consensus.cc:2454] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37bf17d7fc54402ca1fa16f816b2b5e2 in term 1.
I20260620 18:44:18.329416 20169 raft_consensus.cc:2454] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37bf17d7fc54402ca1fa16f816b2b5e2 in term 1.
I20260620 18:44:18.329584 19834 leader_election.cc:302] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37bf17d7fc54402ca1fa16f816b2b5e2, 90572ad288ce47b9a702cef4292ab94a; no voters:
I20260620 18:44:18.329703 20241 raft_consensus.cc:2781] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:18.329886 20241 raft_consensus.cc:686] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 1 LEADER]: Becoming Leader. State: Replica: 37bf17d7fc54402ca1fa16f816b2b5e2, State: Running, Role: LEADER
I20260620 18:44:18.329999 20241 consensus_queue.cc:237] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35977 } } peers { permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 } } peers { permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 } }
I20260620 18:44:18.330909 19757 catalog_manager.cc:5495] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37bf17d7fc54402ca1fa16f816b2b5e2 (127.15.77.65). New cstate: current_term: 1 leader_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37bf17d7fc54402ca1fa16f816b2b5e2" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 } health_report { overall_health: UNKNOWN } } }
W20260620 18:44:18.339798 20082 tablet.cc:2369] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:44:18.340050 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.337648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.15.77.65:35977)
0620 18:44:18.337714 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.337716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.337725 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:44:18.337760 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.337761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.337777 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.337842 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:18.337968 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.338425 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.339186 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:18.339436 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:18.339501 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:18.339668 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:18.339802 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:18.339895 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.339939 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.339962 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 18:44:18.341974 20035 raft_consensus.cc:1261] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a [term 1 FOLLOWER]: Refusing update from remote peer 37bf17d7fc54402ca1fa16f816b2b5e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:18.342351 20169 raft_consensus.cc:1261] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d [term 1 FOLLOWER]: Refusing update from remote peer 37bf17d7fc54402ca1fa16f816b2b5e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:18.342361 20244 consensus_queue.cc:1035] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90572ad288ce47b9a702cef4292ab94a" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:18.342649 20244 consensus_queue.cc:1035] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f8e480bc1154abc9aaf562d79954e6d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 40581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:18.345000 20248 mvcc.cc:205] Tried to move back new op lower bound from 7298994414965616640 to 7298994414920380416. Current Snapshot: MvccSnapshot[applied={T|T < 7298994414965616640}]
I20260620 18:44:18.345338 20247 mvcc.cc:205] Tried to move back new op lower bound from 7298994414965616640 to 7298994414920380416. Current Snapshot: MvccSnapshot[applied={T|T < 7298994414965616640}]
I20260620 18:44:18.346807 20250 mvcc.cc:205] Tried to move back new op lower bound from 7298994414965616640 to 7298994414920380416. Current Snapshot: MvccSnapshot[applied={T|T < 7298994414965616640}]
I20260620 18:44:18.347780 19757 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:18.347872 19757 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:18.348662 19757 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b68f7a17e3c7491ba3971a71ed76f0ac: Sending DeleteTablet for 3 replicas of tablet af1ac1505e7f41c7be19effd5c31c83f
I20260620 18:44:18.348974 19881 tablet_service.cc:1514] Processing DeleteTablet for tablet af1ac1505e7f41c7be19effd5c31c83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:18 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:53840
I20260620 18:44:18.349011 20015 tablet_service.cc:1514] Processing DeleteTablet for tablet af1ac1505e7f41c7be19effd5c31c83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:18 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:33626
I20260620 18:44:18.349200 20256 tablet_replica.cc:330] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2: stopping tablet replica
I20260620 18:44:18.349210 20149 tablet_service.cc:1514] Processing DeleteTablet for tablet af1ac1505e7f41c7be19effd5c31c83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:18 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:51334
I20260620 18:44:18.349308 20256 raft_consensus.cc:2229] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:44:18.349407 20256 raft_consensus.cc:2258] T af1ac1505e7f41c7be19effd5c31c83f P 37bf17d7fc54402ca1fa16f816b2b5e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:18.349404 20257 tablet_replica.cc:330] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: stopping tablet replica
I20260620 18:44:18.349478 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 19819
I20260620 18:44:18.349498 20257 raft_consensus.cc:2229] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:18.349606 20257 raft_consensus.cc:2258] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:18.350183 20258 tablet_replica.cc:330] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: stopping tablet replica
I20260620 18:44:18.350283 20257 ts_tablet_manager.cc:1905] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:18.350284 20258 raft_consensus.cc:2229] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:18.350387 20258 raft_consensus.cc:2258] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:18.350845 20258 ts_tablet_manager.cc:1905] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:18.351948 20257 ts_tablet_manager.cc:1918] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:18.352002 20257 log.cc:1196] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/af1ac1505e7f41c7be19effd5c31c83f
I20260620 18:44:18.352231 20257 ts_tablet_manager.cc:1939] T af1ac1505e7f41c7be19effd5c31c83f P 90572ad288ce47b9a702cef4292ab94a: Deleting consensus metadata
I20260620 18:44:18.352669 19741 catalog_manager.cc:4841] TS 90572ad288ce47b9a702cef4292ab94a (127.15.77.66:44955): tablet af1ac1505e7f41c7be19effd5c31c83f (table test-table [id=74a692bd4d6443c6a46ea76d796ab90d]) successfully deleted
I20260620 18:44:18.352792 20258 ts_tablet_manager.cc:1918] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:18.352856 20258 log.cc:1196] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/af1ac1505e7f41c7be19effd5c31c83f
I20260620 18:44:18.353127 20258 ts_tablet_manager.cc:1939] T af1ac1505e7f41c7be19effd5c31c83f P 8f8e480bc1154abc9aaf562d79954e6d: Deleting consensus metadata
I20260620 18:44:18.353544 19742 catalog_manager.cc:4841] TS 8f8e480bc1154abc9aaf562d79954e6d (127.15.77.67:40581): tablet af1ac1505e7f41c7be19effd5c31c83f (table test-table [id=74a692bd4d6443c6a46ea76d796ab90d]) successfully deleted
W20260620 18:44:18.356915 19741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:35977 (error 108)
I20260620 18:44:18.357129 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 19953
I20260620 18:44:18.359138 20231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.358963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:35977 (local address 127.0.0.1:53868)
0620 18:44:18.359030 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.359100 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 18:44:18.359279 19741 catalog_manager.cc:4584] TS 37bf17d7fc54402ca1fa16f816b2b5e2 (127.15.77.65:35977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af1ac1505e7f41c7be19effd5c31c83f: Network error: Client connection negotiation failed: client connection to 127.15.77.65:35977: connect: Connection refused (error 111)
I20260620 18:44:18.362185 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 20087
I20260620 18:44:18.366870 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 19726
2026-06-20T18:44:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5941 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-20T18:44:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:18Z Disabled control of system clock
I20260620 18:44:18.394555 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40141
--webserver_interface=127.15.77.126
--webserver_port=0
--builtin_ntp_servers=127.15.77.84:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 18:44:18.476739 20263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:18.476964 20263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:18.476986 20263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:18.478067 20263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:18.478102 20263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:18.478116 20263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:18.478127 20263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:18.478147 20263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:18.479597 20263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:18.479811 20263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:18.480058 20263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:18.484848 20271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:18.485024 20263 server_base.cc:1034] running on GCE node
W20260620 18:44:18.484819 20269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:18.484826 20268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:18.485445 20263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:18.485790 20263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:18.486963 20263 hybrid_clock.cc:648] HybridClock initialized: now 1781981058486956 us; error 50 us; skew 500 ppm
I20260620 18:44:18.488358 20263 webserver.cc:460] Webserver started at http://127.15.77.126:38829/ using document root <none> and password file <none>
I20260620 18:44:18.488621 20263 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:18.488730 20263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:18.488875 20263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:18.489987 20263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "7f3df4eae42949a7857829752ee7ef99"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "3e6f683beb25e5bf16c55365631534ae"
server_key_iv: "685c1e4367944aa54cce1e3940c9df05"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.490363 20263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "7f3df4eae42949a7857829752ee7ef99"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "3e6f683beb25e5bf16c55365631534ae"
server_key_iv: "685c1e4367944aa54cce1e3940c9df05"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.491773 20263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:44:18.492525 20277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.492748 20263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:18.492854 20263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "7f3df4eae42949a7857829752ee7ef99"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "3e6f683beb25e5bf16c55365631534ae"
server_key_iv: "685c1e4367944aa54cce1e3940c9df05"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.492938 20263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:18.496371 20263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:18.496649 20263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:18.496794 20263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:18.500937 20263 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:40141
I20260620 18:44:18.500968 20329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:40141 every 8 connection(s)
I20260620 18:44:18.501343 20263 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:18.501986 20330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:18.504824 20330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99: Bootstrap starting.
I20260620 18:44:18.505357 20330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:18.505602 20330 log.cc:824] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:18.506237 20330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99: No bootstrap required, opened a new log
I20260620 18:44:18.507894 20330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3df4eae42949a7857829752ee7ef99" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40141 } }
I20260620 18:44:18.508034 20330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:18.508059 20330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f3df4eae42949a7857829752ee7ef99, State: Initialized, Role: FOLLOWER
I20260620 18:44:18.508198 20330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3df4eae42949a7857829752ee7ef99" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40141 } }
I20260620 18:44:18.508279 20330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:18.508302 20330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:18.508332 20330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:18.508843 20330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3df4eae42949a7857829752ee7ef99" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40141 } }
I20260620 18:44:18.508929 20330 leader_election.cc:302] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f3df4eae42949a7857829752ee7ef99; no voters:
I20260620 18:44:18.509109 20330 leader_election.cc:288] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:18.509189 20333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:18.509215 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 20263
I20260620 18:44:18.509306 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:18.509356 20333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [term 1 LEADER]: Becoming Leader. State: Replica: 7f3df4eae42949a7857829752ee7ef99, State: Running, Role: LEADER
I20260620 18:44:18.509416 20330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:18.509490 15669 external_mini_cluster.cc:1428] Setting key 14454211c10fcf953cef794f493f1e84
I20260620 18:44:18.509454 20333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3df4eae42949a7857829752ee7ef99" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40141 } }
I20260620 18:44:18.510283 20334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f3df4eae42949a7857829752ee7ef99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3df4eae42949a7857829752ee7ef99" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40141 } } }
I20260620 18:44:18.510366 20334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:18.510586 20335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f3df4eae42949a7857829752ee7ef99. Latest consensus state: current_term: 1 leader_uuid: "7f3df4eae42949a7857829752ee7ef99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3df4eae42949a7857829752ee7ef99" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40141 } } }
I20260620 18:44:18.510975 20335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99 [sys.catalog]: This master's current role is: LEADER
W20260620 18:44:18.511785 20351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:44:18.511847 20351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:44:18.511900 20344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:18.512204 20344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:18.512210 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.509956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53014 (local address 127.15.77.126:40141)
0620 18:44:18.510191 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.510203 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.510243 (+ 40us) server_negotiation.cc:407] Connection header received
0620 18:44:18.510368 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.510375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.510427 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.511114 (+ 687us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.511681 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.511684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.511684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:18.511686 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.511713 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:18.511722 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.511806 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.512002 (+ 196us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.512068 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260620 18:44:18.513810 20344 catalog_manager.cc:1347] Generated new cluster ID: b2706910d6df466582c4a90b59882b95
I20260620 18:44:18.513868 20344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:44:18.521217 20344 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:18.521399 20344 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:18.534267 20344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99: Generated new TSK 0
I20260620 18:44:18.534508 20344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:44:18.537890 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40141
--builtin_ntp_servers=127.15.77.84:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 18:44:18.621361 20354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:18.621613 20354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:18.621652 20354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:18.622864 20354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:18.622932 20354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:18.622977 20354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:44:18.624444 20354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:18.624718 20354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:18.625026 20354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:18.630156 20360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:18.630156 20359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:18.630249 20362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:18.630725 20354 server_base.cc:1034] running on GCE node
I20260620 18:44:18.630961 20354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:18.631269 20354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:18.632491 20354 hybrid_clock.cc:648] HybridClock initialized: now 1781981058632468 us; error 35 us; skew 500 ppm
I20260620 18:44:18.634096 20354 webserver.cc:460] Webserver started at http://127.15.77.65:44861/ using document root <none> and password file <none>
I20260620 18:44:18.634382 20354 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:18.634454 20354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:18.634604 20354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:18.636209 20354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "880dbca83b2c4256b0c89adeaa925c8e"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "19cc8fcc66d461be33ebeb249c65654a"
server_key_iv: "3b25a25b5b50edafd548987138263070"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.636672 20354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "880dbca83b2c4256b0c89adeaa925c8e"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "19cc8fcc66d461be33ebeb249c65654a"
server_key_iv: "3b25a25b5b50edafd548987138263070"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.638556 20354 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260620 18:44:18.639559 20368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.639775 20354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:18.639880 20354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "880dbca83b2c4256b0c89adeaa925c8e"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "19cc8fcc66d461be33ebeb249c65654a"
server_key_iv: "3b25a25b5b50edafd548987138263070"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.639961 20354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:18.643251 20354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:18.643563 20354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:18.643715 20354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:18.643945 20354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:18.644290 20354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:18.644349 20354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.644389 20354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:18.644416 20354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.649892 20354 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:33929
I20260620 18:44:18.650007 20481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:33929 every 8 connection(s)
I20260620 18:44:18.650332 20354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:44:18.653156 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 20354
I20260620 18:44:18.653270 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:44:18.653443 15669 external_mini_cluster.cc:1428] Setting key 33e6a5e64cfe4b9419c1c10eb64f4f60
I20260620 18:44:18.654086 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.651170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:55869 (local address 127.15.77.126:40141)
0620 18:44:18.651250 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.651253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.651603 (+ 350us) server_negotiation.cc:407] Connection header received
0620 18:44:18.652344 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.652345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.652351 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.652385 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.653795 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.653797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.653797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:18.653802 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.653818 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:18.653822 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.653875 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.653981 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.653995 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 18:44:18.654294 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.651145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40141 (local address 127.15.77.65:55869)
0620 18:44:18.651425 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.651485 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.652220 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.652461 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.652466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.652477 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:18.653608 (+ 1131us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:44:18.653609 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:18.653661 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:44:18.653704 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:18.653895 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:18.653900 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:18.653903 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.653991 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.654061 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":103,"threads_started":1}
I20260620 18:44:18.654505 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40141
--builtin_ntp_servers=127.15.77.84:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260620 18:44:18.654759 20482 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40141
I20260620 18:44:18.654855 20482 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:18.655014 20482 heartbeater.cc:511] Master 127.15.77.126:40141 requested a full tablet report, sending...
I20260620 18:44:18.655665 20294 ts_manager.cc:194] Registered new tserver with Master: 880dbca83b2c4256b0c89adeaa925c8e (127.15.77.65:33929)
W20260620 18:44:18.737195 20485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:18.737407 20485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:18.737430 20485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:18.738641 20485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:18.738684 20485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:18.738741 20485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:44:18.740264 20485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:18.740528 20485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:18.740839 20485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:18.746143 20491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:18.746155 20493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:18.746155 20490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:18.746389 20485 server_base.cc:1034] running on GCE node
I20260620 18:44:18.746665 20485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:18.746922 20485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:18.748088 20485 hybrid_clock.cc:648] HybridClock initialized: now 1781981058748077 us; error 45 us; skew 500 ppm
I20260620 18:44:18.749361 20485 webserver.cc:460] Webserver started at http://127.15.77.66:46451/ using document root <none> and password file <none>
I20260620 18:44:18.749688 20485 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:18.749761 20485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:18.749923 20485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:18.751477 20485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "99613bee52d9480bb25bbafa916a3660"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "77d650efe12d3a6c33a904ea29f15a3f"
server_key_iv: "5d8756f5d3413ae09dfe814c03449021"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.751951 20485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "99613bee52d9480bb25bbafa916a3660"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "77d650efe12d3a6c33a904ea29f15a3f"
server_key_iv: "5d8756f5d3413ae09dfe814c03449021"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.753679 20485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:44:18.754421 20499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.754603 20485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:18.754704 20485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "99613bee52d9480bb25bbafa916a3660"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "77d650efe12d3a6c33a904ea29f15a3f"
server_key_iv: "5d8756f5d3413ae09dfe814c03449021"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.754782 20485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:18.758172 20485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:18.758440 20485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:18.758576 20485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:18.758802 20485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:18.759153 20485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:18.759207 20485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.759248 20485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:18.759275 20485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.765244 20485 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:39155
I20260620 18:44:18.765305 20612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:39155 every 8 connection(s)
I20260620 18:44:18.765661 20485 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:44:18.768863 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.766401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:33887 (local address 127.15.77.126:40141)
0620 18:44:18.766473 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.766475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.766736 (+ 261us) server_negotiation.cc:407] Connection header received
0620 18:44:18.767274 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.767275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.767282 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.767370 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.768548 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.768549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.768550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:18.768555 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.768574 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:18.768579 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.768660 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.768771 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.768790 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 18:44:18.769079 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.766393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40141 (local address 127.15.77.66:33887)
0620 18:44:18.766611 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.766647 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.767161 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.767450 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.767454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.767461 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:18.768380 (+ 919us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:44:18.768380 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:18.768409 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:44:18.768438 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:18.768669 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:18.768676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:18.768679 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.768768 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.768836 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260620 18:44:18.769634 20613 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40141
I20260620 18:44:18.769712 20613 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:18.769873 20613 heartbeater.cc:511] Master 127.15.77.126:40141 requested a full tablet report, sending...
I20260620 18:44:18.770211 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 20485
I20260620 18:44:18.770329 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:44:18.770287 20294 ts_manager.cc:194] Registered new tserver with Master: 99613bee52d9480bb25bbafa916a3660 (127.15.77.66:39155)
I20260620 18:44:18.770550 15669 external_mini_cluster.cc:1428] Setting key 5dfc7ac5cb07104619832ec003db7015
I20260620 18:44:18.771389 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40141
--builtin_ntp_servers=127.15.77.84:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 18:44:18.853785 20616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:18.853996 20616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:18.854020 20616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:18.855228 20616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:18.855270 20616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:18.855297 20616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:44:18.856854 20616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:18.857112 20616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:18.857468 20616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:18.862736 20622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:18.862701 20624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:18.862701 20621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:18.863087 20616 server_base.cc:1034] running on GCE node
I20260620 18:44:18.863395 20616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:18.863677 20616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:18.864846 20616 hybrid_clock.cc:648] HybridClock initialized: now 1781981058864838 us; error 42 us; skew 500 ppm
I20260620 18:44:18.866211 20616 webserver.cc:460] Webserver started at http://127.15.77.67:43317/ using document root <none> and password file <none>
I20260620 18:44:18.866479 20616 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:18.866554 20616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:18.866714 20616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:18.867887 20616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "a137be6c31104c52b833d4958a55c4df"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "efc583bb55d8501b1867e205c3f61ad7"
server_key_iv: "eba6f73c81a94140fd5ff9dfb136af62"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.868261 20616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "a137be6c31104c52b833d4958a55c4df"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "efc583bb55d8501b1867e205c3f61ad7"
server_key_iv: "eba6f73c81a94140fd5ff9dfb136af62"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.869669 20616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:44:18.870384 20630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.870550 20616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:18.870653 20616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "a137be6c31104c52b833d4958a55c4df"
format_stamp: "Formatted at 2026-06-20 18:44:18 on dist-test-slave-tvl9"
server_key: "efc583bb55d8501b1867e205c3f61ad7"
server_key_iv: "eba6f73c81a94140fd5ff9dfb136af62"
server_key_version: "encryptionkey@0"
I20260620 18:44:18.870728 20616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:18.874154 20616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:18.874424 20616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:18.874583 20616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:18.874809 20616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:18.875121 20616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:18.875177 20616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.875217 20616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:18.875244 20616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:18.881071 20616 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:33031
I20260620 18:44:18.881114 20743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:33031 every 8 connection(s)
I20260620 18:44:18.881466 20616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:44:18.884601 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.882235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:43711 (local address 127.15.77.126:40141)
0620 18:44:18.882328 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.882331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.882550 (+ 219us) server_negotiation.cc:407] Connection header received
0620 18:44:18.883026 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.883027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.883035 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.883093 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.884340 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.884341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.884342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:18.884346 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.884360 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:18.884364 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.884420 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.884530 (+ 110us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.884544 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 18:44:18.884850 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.882213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40141 (local address 127.15.77.67:43711)
0620 18:44:18.882426 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.882461 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.882935 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.883166 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.883169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.883178 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:18.884171 (+ 993us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:44:18.884172 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:18.884203 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:44:18.884235 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:18.884449 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:18.884455 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:18.884458 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.884537 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.884616 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":113,"threads_started":1}
I20260620 18:44:18.885380 20744 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40141
I20260620 18:44:18.885463 20744 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:18.885641 20744 heartbeater.cc:511] Master 127.15.77.126:40141 requested a full tablet report, sending...
I20260620 18:44:18.886039 20294 ts_manager.cc:194] Registered new tserver with Master: a137be6c31104c52b833d4958a55c4df (127.15.77.67:33031)
I20260620 18:44:18.886297 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 20616
I20260620 18:44:18.886399 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:44:18.886617 15669 external_mini_cluster.cc:1428] Setting key c5efa9917ff27a31324dc82fe9dc30fd
I20260620 18:44:18.887470 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:44:18.889824 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.888723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53018 (local address 127.15.77.126:40141)
0620 18:44:18.888777 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.888779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.888908 (+ 129us) server_negotiation.cc:407] Connection header received
0620 18:44:18.888989 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.888990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.888996 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.889045 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.889591 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.889593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.889594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:18.889598 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.889612 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:18.889616 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.889681 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.889735 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.889751 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 18:44:18.891252 20294 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:44:18.893579 20294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:18.898378 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.896559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:33031 (local address 127.0.0.1:49568)
0620 18:44:18.896727 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.896766 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.896847 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.897206 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.897213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.897221 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:18.897935 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:44:18.897936 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:18.897955 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:44:18.897980 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:18.898218 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:18.898225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:18.898230 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.898283 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.898299 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":60,"threads_started":1}
I20260620 18:44:18.898721 20753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.896704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.15.77.67:33031)
0620 18:44:18.896938 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.896944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.896954 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:18.896983 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.896985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.896995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.897145 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.898044 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.898052 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.898052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:18.898057 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.898107 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:18.898132 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.898602 (+ 470us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.898628 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.898645 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":107,"threads_started":1}
I20260620 18:44:18.899494 20678 tablet_service.cc:1467] Processing CreateTablet for tablet d2f03e88f4df4743ab5b770c7104bfc5 (DEFAULT_TABLE table=test-table [id=e4f22c0b9f19415883fc4b2292992ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:18.899915 20678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f03e88f4df4743ab5b770c7104bfc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:18.900031 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.897750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33794 (local address 127.15.77.66:39155)
0620 18:44:18.897957 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.897963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.897975 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:44:18.898003 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.898006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.898016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.898090 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.898836 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.898840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.898840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:18.898845 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.898877 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:18.898892 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.899025 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.899925 (+ 900us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.899951 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":77,"threads_started":1}
I20260620 18:44:18.900561 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.897222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:39155 (local address 127.0.0.1:33794)
0620 18:44:18.897498 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.897510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.897548 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.898104 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.898105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.898108 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:18.898577 (+ 469us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:44:18.898578 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:18.898586 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:44:18.898627 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:18.899820 (+ 1193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:18.899821 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:18.899822 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.899912 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.899926 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":84,"threads_started":1}
I20260620 18:44:18.901881 20547 tablet_service.cc:1467] Processing CreateTablet for tablet d2f03e88f4df4743ab5b770c7104bfc5 (DEFAULT_TABLE table=test-table [id=e4f22c0b9f19415883fc4b2292992ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:18.902228 20547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f03e88f4df4743ab5b770c7104bfc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:18.902455 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.897398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33929 (local address 127.0.0.1:40258)
0620 18:44:18.897919 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:18.897930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:18.897976 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:18.901335 (+ 3359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:18.901336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:18.901338 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:18.901829 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:44:18.901829 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:18.901837 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:44:18.901850 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:18.902309 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:18.902310 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:18.902311 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 18:44:18.902374 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 18:44:18.902386 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":176,"threads_started":1}
I20260620 18:44:18.902840 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:18.897737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.15.77.65:33929)
0620 18:44:18.899108 (+ 1371us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:18.899112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:18.899122 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:18.899146 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:18.899148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:18.899156 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:18.899224 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:18.902143 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:18.902152 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:18.902152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:18.902157 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:18.902192 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:18.902221 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:18.902283 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:18.902719 (+ 436us) server_negotiation.cc:299] Negotiation successful
0620 18:44:18.902740 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321,"thread_start_us":73,"threads_started":1}
I20260620 18:44:18.903470 20758 tablet_bootstrap.cc:492] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: Bootstrap starting.
I20260620 18:44:18.903476 20416 tablet_service.cc:1467] Processing CreateTablet for tablet d2f03e88f4df4743ab5b770c7104bfc5 (DEFAULT_TABLE table=test-table [id=e4f22c0b9f19415883fc4b2292992ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:18.903827 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f03e88f4df4743ab5b770c7104bfc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:18.904125 20758 tablet_bootstrap.cc:654] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:18.904604 20758 log.cc:824] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:18.905483 20758 tablet_bootstrap.cc:492] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: No bootstrap required, opened a new log
I20260620 18:44:18.905547 20758 ts_tablet_manager.cc:1397] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:44:18.906352 20760 tablet_bootstrap.cc:492] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: Bootstrap starting.
I20260620 18:44:18.907030 20760 tablet_bootstrap.cc:654] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:18.907490 20760 log.cc:824] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:18.908349 20760 tablet_bootstrap.cc:492] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: No bootstrap required, opened a new log
I20260620 18:44:18.908401 20760 ts_tablet_manager.cc:1397] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:44:18.908285 20758 raft_consensus.cc:348] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 } } peers { permanent_uuid: "99613bee52d9480bb25bbafa916a3660" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39155 } } peers { permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 } }
I20260620 18:44:18.908445 20758 raft_consensus.cc:374] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:18.908484 20758 raft_consensus.cc:729] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a137be6c31104c52b833d4958a55c4df, State: Initialized, Role: FOLLOWER
I20260620 18:44:18.908667 20758 consensus_queue.cc:260] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 } } peers { permanent_uuid: "99613bee52d9480bb25bbafa916a3660" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39155 } } peers { permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 } }
I20260620 18:44:18.908982 20758 ts_tablet_manager.cc:1428] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260620 18:44:18.909019 20744 heartbeater.cc:503] Master 127.15.77.126:40141 was elected leader, sending a full tablet report...
I20260620 18:44:18.909185 20761 tablet_bootstrap.cc:492] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e: Bootstrap starting.
I20260620 18:44:18.909860 20761 tablet_bootstrap.cc:654] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:18.910188 20761 log.cc:824] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:18.910943 20760 raft_consensus.cc:348] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 } } peers { permanent_uuid: "99613bee52d9480bb25bbafa916a3660" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39155 } } peers { permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 } }
I20260620 18:44:18.911126 20760 raft_consensus.cc:374] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:18.911185 20760 raft_consensus.cc:729] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99613bee52d9480bb25bbafa916a3660, State: Initialized, Role: FOLLOWER
I20260620 18:44:18.911334 20760 consensus_queue.cc:260] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 } } peers { permanent_uuid: "99613bee52d9480bb25bbafa916a3660" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39155 } } peers { permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 } }
I20260620 18:44:18.911403 20761 tablet_bootstrap.cc:492] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e: No bootstrap required, opened a new log
I20260620 18:44:18.911562 20761 ts_tablet_manager.cc:1397] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:44:18.911648 20613 heartbeater.cc:503] Master 127.15.77.126:40141 was elected leader, sending a full tablet report...
I20260620 18:44:18.911638 20760 ts_tablet_manager.cc:1428] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:44:18.913870 20761 raft_consensus.cc:348] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 } } peers { permanent_uuid: "99613bee52d9480bb25bbafa916a3660" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39155 } } peers { permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 } }
I20260620 18:44:18.914103 20761 raft_consensus.cc:374] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:18.914170 20761 raft_consensus.cc:729] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880dbca83b2c4256b0c89adeaa925c8e, State: Initialized, Role: FOLLOWER
I20260620 18:44:18.914286 20761 consensus_queue.cc:260] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 } } peers { permanent_uuid: "99613bee52d9480bb25bbafa916a3660" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39155 } } peers { permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 } }
I20260620 18:44:18.914549 20761 ts_tablet_manager.cc:1428] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:44:18.914572 20482 heartbeater.cc:503] Master 127.15.77.126:40141 was elected leader, sending a full tablet report...
W20260620 18:44:19.016265 20614 tablet.cc:2369] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:44:19.021061 20765 raft_consensus.cc:482] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:19.021232 20765 raft_consensus.cc:504] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 } } peers { permanent_uuid: "99613bee52d9480bb25bbafa916a3660" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39155 } } peers { permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 } }
I20260620 18:44:19.021652 20765 leader_election.cc:288] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a137be6c31104c52b833d4958a55c4df (127.15.77.67:33031), 880dbca83b2c4256b0c89adeaa925c8e (127.15.77.65:33929)
I20260620 18:44:19.022979 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.021845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:33031 (local address 127.15.77.66:38211)
0620 18:44:19.021917 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:19.021930 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:19.021971 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:19.022157 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:19.022157 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:19.022160 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:19.022638 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:44:19.022638 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:19.022646 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:44:19.022659 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:19.022838 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:19.022839 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:19.022840 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 18:44:19.022893 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 18:44:19.022906 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 18:44:19.023039 20753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.021916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:38211 (local address 127.15.77.67:33031)
0620 18:44:19.021979 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.021982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.021994 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:44:19.022027 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.022028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.022034 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.022115 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:19.022742 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:19.022743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:19.022744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:19.022748 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:19.022772 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:19.022776 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:19.022842 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:19.022921 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 18:44:19.022938 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 18:44:19.023195 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.021954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:36077 (local address 127.15.77.65:33929)
0620 18:44:19.022000 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.022002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.022109 (+ 107us) server_negotiation.cc:407] Connection header received
0620 18:44:19.022175 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.022176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.022181 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.022242 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:19.022926 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:19.022927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:19.022927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:19.022931 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:19.022946 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:19.022950 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:19.023003 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:19.023103 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 18:44:19.023119 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:44:19.023406 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.021858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33929 (local address 127.15.77.66:36077)
0620 18:44:19.022038 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:19.022048 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:19.022108 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:19.022258 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:19.022258 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:19.022260 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:19.022816 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:44:19.022816 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:19.022822 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:44:19.022834 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:19.023012 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:19.023013 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:19.023013 (+ 0us) client_negotiation.cc:769] Sending connection context
0620 18:44:19.023288 (+ 275us) client_negotiation.cc:240] Negotiation successful
0620 18:44:19.023304 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":102,"threads_started":1}
I20260620 18:44:19.023409 20698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f03e88f4df4743ab5b770c7104bfc5" candidate_uuid: "99613bee52d9480bb25bbafa916a3660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a137be6c31104c52b833d4958a55c4df" is_pre_election: true
I20260620 18:44:19.023603 20698 raft_consensus.cc:2454] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99613bee52d9480bb25bbafa916a3660 in term 0.
I20260620 18:44:19.023746 20436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f03e88f4df4743ab5b770c7104bfc5" candidate_uuid: "99613bee52d9480bb25bbafa916a3660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880dbca83b2c4256b0c89adeaa925c8e" is_pre_election: true
I20260620 18:44:19.023857 20500 leader_election.cc:302] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99613bee52d9480bb25bbafa916a3660, a137be6c31104c52b833d4958a55c4df; no voters:
I20260620 18:44:19.023895 20436 raft_consensus.cc:2454] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99613bee52d9480bb25bbafa916a3660 in term 0.
I20260620 18:44:19.023979 20765 raft_consensus.cc:2781] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:19.024065 20765 raft_consensus.cc:482] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:19.024113 20765 raft_consensus.cc:3037] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:19.024832 20765 raft_consensus.cc:504] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 } } peers { permanent_uuid: "99613bee52d9480bb25bbafa916a3660" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39155 } } peers { permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 } }
I20260620 18:44:19.025015 20765 leader_election.cc:288] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [CANDIDATE]: Term 1 election: Requested vote from peers a137be6c31104c52b833d4958a55c4df (127.15.77.67:33031), 880dbca83b2c4256b0c89adeaa925c8e (127.15.77.65:33929)
I20260620 18:44:19.025125 20698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f03e88f4df4743ab5b770c7104bfc5" candidate_uuid: "99613bee52d9480bb25bbafa916a3660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a137be6c31104c52b833d4958a55c4df"
I20260620 18:44:19.025208 20698 raft_consensus.cc:3037] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:19.025213 20436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f03e88f4df4743ab5b770c7104bfc5" candidate_uuid: "99613bee52d9480bb25bbafa916a3660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880dbca83b2c4256b0c89adeaa925c8e"
I20260620 18:44:19.025293 20436 raft_consensus.cc:3037] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:19.025954 20698 raft_consensus.cc:2454] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99613bee52d9480bb25bbafa916a3660 in term 1.
I20260620 18:44:19.026095 20436 raft_consensus.cc:2454] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99613bee52d9480bb25bbafa916a3660 in term 1.
I20260620 18:44:19.026161 20500 leader_election.cc:302] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99613bee52d9480bb25bbafa916a3660, a137be6c31104c52b833d4958a55c4df; no voters:
I20260620 18:44:19.026249 20765 raft_consensus.cc:2781] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:19.026391 20765 raft_consensus.cc:686] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 1 LEADER]: Becoming Leader. State: Replica: 99613bee52d9480bb25bbafa916a3660, State: Running, Role: LEADER
I20260620 18:44:19.026518 20765 consensus_queue.cc:237] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 } } peers { permanent_uuid: "99613bee52d9480bb25bbafa916a3660" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39155 } } peers { permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 } }
I20260620 18:44:19.027511 20293 catalog_manager.cc:5495] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99613bee52d9480bb25bbafa916a3660 (127.15.77.66). New cstate: current_term: 1 leader_uuid: "99613bee52d9480bb25bbafa916a3660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99613bee52d9480bb25bbafa916a3660" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 } health_report { overall_health: UNKNOWN } } }
I20260620 18:44:19.049197 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.048247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.15.77.66:39155)
0620 18:44:19.048309 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.048312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.048327 (+ 15us) server_negotiation.cc:407] Connection header received
0620 18:44:19.048364 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.048365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.048377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.048427 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:19.048949 (+ 522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:19.048951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:19.048951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:19.048955 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:19.048971 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:19.048975 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:19.049022 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:19.049064 (+ 42us) server_negotiation.cc:299] Negotiation successful
0620 18:44:19.049085 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 18:44:19.051129 20698 raft_consensus.cc:1261] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df [term 1 FOLLOWER]: Refusing update from remote peer 99613bee52d9480bb25bbafa916a3660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:19.051128 20436 raft_consensus.cc:1261] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e [term 1 FOLLOWER]: Refusing update from remote peer 99613bee52d9480bb25bbafa916a3660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:19.051532 20765 consensus_queue.cc:1035] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a137be6c31104c52b833d4958a55c4df" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:19.051718 20768 consensus_queue.cc:1035] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880dbca83b2c4256b0c89adeaa925c8e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:19.054342 20772 mvcc.cc:205] Tried to move back new op lower bound from 7298994417870405632 to 7298994417773350912. Current Snapshot: MvccSnapshot[applied={T|T < 7298994417870405632}]
I20260620 18:44:19.057556 20294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:19.057693 20294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:19.058570 20294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f3df4eae42949a7857829752ee7ef99: Sending DeleteTablet for 3 replicas of tablet d2f03e88f4df4743ab5b770c7104bfc5
I20260620 18:44:19.058957 20678 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f03e88f4df4743ab5b770c7104bfc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:19 UTC) from {username='slave'} at 127.0.0.1:49568
I20260620 18:44:19.059024 20416 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f03e88f4df4743ab5b770c7104bfc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:19 UTC) from {username='slave'} at 127.0.0.1:40258
I20260620 18:44:19.059217 20547 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f03e88f4df4743ab5b770c7104bfc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:19 UTC) from {username='slave'} at 127.0.0.1:33794
I20260620 18:44:19.059387 20782 tablet_replica.cc:330] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e: stopping tablet replica
I20260620 18:44:19.059476 20783 tablet_replica.cc:330] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: stopping tablet replica
I20260620 18:44:19.059576 20783 raft_consensus.cc:2229] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:44:19.059607 20782 raft_consensus.cc:2229] T d2f03e88f4df4743ab5b770c7104bfc5 P 880dbca83b2c4256b0c89adeaa925c8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:19.059727 20783 raft_consensus.cc:2258] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:19.059819 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 20354
I20260620 18:44:19.060606 20783 ts_tablet_manager.cc:1905] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:19.062415 20783 ts_tablet_manager.cc:1918] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:19.062480 20783 log.cc:1196] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/d2f03e88f4df4743ab5b770c7104bfc5
I20260620 18:44:19.062767 20783 ts_tablet_manager.cc:1939] T d2f03e88f4df4743ab5b770c7104bfc5 P 99613bee52d9480bb25bbafa916a3660: Deleting consensus metadata
I20260620 18:44:19.063202 20279 catalog_manager.cc:4841] TS 99613bee52d9480bb25bbafa916a3660 (127.15.77.66:39155): tablet d2f03e88f4df4743ab5b770c7104bfc5 (table test-table [id=e4f22c0b9f19415883fc4b2292992ef0]) successfully deleted
W20260620 18:44:19.066274 20279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:33929 (error 108)
I20260620 18:44:19.066406 20781 tablet_replica.cc:330] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: stopping tablet replica
I20260620 18:44:19.066555 20781 raft_consensus.cc:2229] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:19.066685 20781 raft_consensus.cc:2258] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:19.066929 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.066764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33929 (local address 127.0.0.1:40266)
0620 18:44:19.066837 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:19.066891 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:33929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 18:44:19.067037 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 20485
W20260620 18:44:19.067054 20279 catalog_manager.cc:4584] TS 880dbca83b2c4256b0c89adeaa925c8e (127.15.77.65:33929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d2f03e88f4df4743ab5b770c7104bfc5: Network error: Client connection negotiation failed: client connection to 127.15.77.65:33929: connect: Connection refused (error 111)
I20260620 18:44:19.067561 20781 ts_tablet_manager.cc:1905] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:19.069145 20781 ts_tablet_manager.cc:1918] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:19.069209 20781 log.cc:1196] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/d2f03e88f4df4743ab5b770c7104bfc5
I20260620 18:44:19.069459 20781 ts_tablet_manager.cc:1939] T d2f03e88f4df4743ab5b770c7104bfc5 P a137be6c31104c52b833d4958a55c4df: Deleting consensus metadata
I20260620 18:44:19.069931 20278 catalog_manager.cc:4841] TS a137be6c31104c52b833d4958a55c4df (127.15.77.67:33031): tablet d2f03e88f4df4743ab5b770c7104bfc5 (table test-table [id=e4f22c0b9f19415883fc4b2292992ef0]) successfully deleted
I20260620 18:44:19.072439 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 20616
I20260620 18:44:19.077662 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 20263
2026-06-20T18:44:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (710 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260620 18:44:19.097409 15669 mini_oidc.cc:152] Starting JWKS server
I20260620 18:44:19.098551 15669 webserver.cc:460] Webserver started at https://127.0.0.1:32887/ using document root <none> and password file <none>
I20260620 18:44:19.098703 15669 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 18:44:19.098896 15669 webserver.cc:460] Webserver started at http://127.0.0.1:37385/ using document root <none> and password file <none>
2026-06-20T18:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:19Z Disabled control of system clock
I20260620 18:44:19.106632 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:43259
--webserver_interface=127.15.77.126
--webserver_port=0
--builtin_ntp_servers=127.15.77.84:40269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:43259
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32887/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/testchainca.cert with env {}
W20260620 18:44:19.187958 20790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:19.188176 20790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:19.188199 20790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:19.189352 20790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:19.189386 20790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:19.189399 20790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:19.189412 20790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:19.189424 20790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 18:44:19.189437 20790 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 18:44:19.189448 20790 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32887/jwks.json
I20260620 18:44:19.190963 20790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:40269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:43259
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:43259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32887/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:19.191160 20790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:19.191344 20790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:19.196095 20798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:19.196067 20795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:19.196067 20796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:19.196372 20790 server_base.cc:1034] running on GCE node
I20260620 18:44:19.196622 20790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:19.196938 20790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:19.198104 20790 hybrid_clock.cc:648] HybridClock initialized: now 1781981059198092 us; error 45 us; skew 500 ppm
I20260620 18:44:19.199410 20790 webserver.cc:460] Webserver started at http://127.15.77.126:37399/ using document root <none> and password file <none>
I20260620 18:44:19.199684 20790 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:19.199756 20790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:19.199908 20790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:19.200994 20790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "a0a08c955c7a4768a1630e7526f2fd6b"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "4db0d0b6e67c377eda26270c3d0a26f5"
server_key_iv: "70d0817b2afc133494e043ff9340393e"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.201359 20790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "a0a08c955c7a4768a1630e7526f2fd6b"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "4db0d0b6e67c377eda26270c3d0a26f5"
server_key_iv: "70d0817b2afc133494e043ff9340393e"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.202646 20790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:44:19.203372 20804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:19.203562 20790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:44:19.203660 20790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "a0a08c955c7a4768a1630e7526f2fd6b"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "4db0d0b6e67c377eda26270c3d0a26f5"
server_key_iv: "70d0817b2afc133494e043ff9340393e"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.203747 20790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:19.211998 20790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:19.212284 20790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:19.212428 20790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:19.216394 20790 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:43259
I20260620 18:44:19.216424 20856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:43259 every 8 connection(s)
I20260620 18:44:19.216765 20790 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:19.217693 20857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:19.220099 20857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b: Bootstrap starting.
I20260620 18:44:19.220758 20857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:19.220990 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 20790
I20260620 18:44:19.221091 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:19.221158 20857 log.cc:824] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:19.221278 15669 external_mini_cluster.cc:1428] Setting key 679afa9ccc561d54f00c0d2617200cdf
I20260620 18:44:19.221942 20857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b: No bootstrap required, opened a new log
I20260620 18:44:19.225286 20857 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a08c955c7a4768a1630e7526f2fd6b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43259 } }
I20260620 18:44:19.225489 20857 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:19.225543 20857 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a08c955c7a4768a1630e7526f2fd6b, State: Initialized, Role: FOLLOWER
I20260620 18:44:19.225714 20857 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a08c955c7a4768a1630e7526f2fd6b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43259 } }
I20260620 18:44:19.225817 20857 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:19.225864 20857 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:19.225916 20857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:19.226686 20857 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a08c955c7a4768a1630e7526f2fd6b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43259 } }
I20260620 18:44:19.226804 20857 leader_election.cc:302] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0a08c955c7a4768a1630e7526f2fd6b; no voters:
I20260620 18:44:19.226980 20857 leader_election.cc:288] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:19.227039 20862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:19.227218 20862 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [term 1 LEADER]: Becoming Leader. State: Replica: a0a08c955c7a4768a1630e7526f2fd6b, State: Running, Role: LEADER
I20260620 18:44:19.227245 20857 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:19.227381 20862 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a08c955c7a4768a1630e7526f2fd6b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43259 } }
I20260620 18:44:19.227806 20863 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0a08c955c7a4768a1630e7526f2fd6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a08c955c7a4768a1630e7526f2fd6b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43259 } } }
I20260620 18:44:19.227914 20863 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:19.228137 20864 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0a08c955c7a4768a1630e7526f2fd6b. Latest consensus state: current_term: 1 leader_uuid: "a0a08c955c7a4768a1630e7526f2fd6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a08c955c7a4768a1630e7526f2fd6b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43259 } } }
I20260620 18:44:19.228204 20864 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:19.228255 20871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:19.228802 20871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:19.232100 20871 catalog_manager.cc:1347] Generated new cluster ID: 5f268a027144470a9ca13392a68682b4
I20260620 18:44:19.232172 20871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:44:19.233345 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.225368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58748 (local address 127.15.77.126:43259)
0620 18:44:19.225623 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.225642 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.225679 (+ 37us) server_negotiation.cc:407] Connection header received
0620 18:44:19.225819 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.225826 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.225912 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.226081 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:19.226330 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.228585 (+ 2255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.230127 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.231057 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.232011 (+ 954us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.232406 (+ 395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:19.232410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:19.232410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:19.232622 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:19.232678 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:19.232696 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:19.232981 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:19.233021 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 18:44:19.233111 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260620 18:44:19.234467 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:44:19.243960 20871 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:19.244675 20871 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:19.251904 20871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0a08c955c7a4768a1630e7526f2fd6b: Generated new TSK 0
I20260620 18:44:19.252110 20871 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 18:44:19.285574 20887 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260620 18:44:19.285693 20887 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260620 18:44:19.286207 20886 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:44:19.262766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:43259 (local address 127.0.0.1:58764)
0620 18:44:19.262952 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:19.262964 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:19.263045 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:19.263219 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:19.263220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:19.263222 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 18:44:19.263292 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:19.263294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.263979 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.263980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:19.264930 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:19.264933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.265297 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.265298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:19.265365 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.265391 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 18:44:19.285833 (+ 20442us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 18:44:19.285946 (+ 113us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.77.126:43259: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260620 18:44:19.286547 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.262885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58764 (local address 127.15.77.126:43259)
0620 18:44:19.262928 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.262931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.263040 (+ 109us) server_negotiation.cc:407] Connection header received
0620 18:44:19.263144 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.263145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.263155 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.263213 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0620 18:44:19.263384 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.263893 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.265019 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.265217 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.265267 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.265513 (+ 246us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 18:44:19.285578 (+ 20065us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 18:44:19.286481 (+ 903us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58764: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 18:44:19.286604 20861 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58764: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.77.126:43259: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260620 18:44:19.286931 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 20790
2026-06-20T18:44:19Z chronyd exiting
I20260620 18:44:19.323173 15669 test_util.cc:175] -----------------------------------------------
I20260620 18:44:19.323254 15669 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781981032911509-15669-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (226 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260620 18:44:19.324046 15669 mini_oidc.cc:152] Starting JWKS server
I20260620 18:44:19.324775 15669 webserver.cc:460] Webserver started at https://127.0.0.1:43715/ using document root <none> and password file <none>
I20260620 18:44:19.324815 15669 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 18:44:19.324983 15669 webserver.cc:460] Webserver started at http://127.0.0.1:42669/ using document root <none> and password file <none>
2026-06-20T18:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:19Z Disabled control of system clock
I20260620 18:44:19.332968 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:43105
--webserver_interface=127.15.77.126
--webserver_port=0
--builtin_ntp_servers=127.15.77.84:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:43105
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43715/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/testchainca.cert with env {}
W20260620 18:44:19.414208 20893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:19.414422 20893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:19.414445 20893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:19.415688 20893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:19.415736 20893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:19.415750 20893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:19.415762 20893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:19.415774 20893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 18:44:19.415787 20893 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 18:44:19.415844 20893 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43715/jwks.json
I20260620 18:44:19.417501 20893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:43105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:43105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43715/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:19.417793 20893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:19.418022 20893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:19.422920 20898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:19.422900 20901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:19.423041 20899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:19.423357 20893 server_base.cc:1034] running on GCE node
I20260620 18:44:19.423581 20893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:19.423830 20893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:19.424973 20893 hybrid_clock.cc:648] HybridClock initialized: now 1781981059424952 us; error 50 us; skew 500 ppm
I20260620 18:44:19.426223 20893 webserver.cc:460] Webserver started at http://127.15.77.126:42659/ using document root <none> and password file <none>
I20260620 18:44:19.426498 20893 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:19.426546 20893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:19.426702 20893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:19.427974 20893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "eaa42482c8d046f5bea4db8e85534f78"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "481559dc3206a2cdb6f670cd5df297f2"
server_key_iv: "db35b7a34962ce53379d38a66fecfc50"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.428346 20893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "eaa42482c8d046f5bea4db8e85534f78"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "481559dc3206a2cdb6f670cd5df297f2"
server_key_iv: "db35b7a34962ce53379d38a66fecfc50"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.429666 20893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:44:19.430413 20907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:19.430605 20893 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:19.430714 20893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "eaa42482c8d046f5bea4db8e85534f78"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "481559dc3206a2cdb6f670cd5df297f2"
server_key_iv: "db35b7a34962ce53379d38a66fecfc50"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.430790 20893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:19.454187 20893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:19.454536 20893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:19.454705 20893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:19.458936 20893 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:43105
I20260620 18:44:19.458976 20959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:43105 every 8 connection(s)
I20260620 18:44:19.459344 20893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:19.460045 20960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:19.462828 20960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78: Bootstrap starting.
I20260620 18:44:19.463482 20960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:19.463733 20960 log.cc:824] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:19.464383 20960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78: No bootstrap required, opened a new log
I20260620 18:44:19.465941 20960 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa42482c8d046f5bea4db8e85534f78" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43105 } }
I20260620 18:44:19.466081 20960 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:19.466104 20960 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaa42482c8d046f5bea4db8e85534f78, State: Initialized, Role: FOLLOWER
I20260620 18:44:19.466251 20960 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa42482c8d046f5bea4db8e85534f78" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43105 } }
I20260620 18:44:19.466338 20960 raft_consensus.cc:388] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:19.466362 20960 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:19.466390 20960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:19.466889 20960 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa42482c8d046f5bea4db8e85534f78" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43105 } }
I20260620 18:44:19.466974 20960 leader_election.cc:302] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eaa42482c8d046f5bea4db8e85534f78; no voters:
I20260620 18:44:19.467096 20960 leader_election.cc:288] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:19.467163 20963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:19.467412 20960 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:19.467494 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 20893
I20260620 18:44:19.467619 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:19.467810 15669 external_mini_cluster.cc:1428] Setting key 623f73f6182c88e79cdc5ae777d8bdd8
I20260620 18:44:19.467964 20963 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [term 1 LEADER]: Becoming Leader. State: Replica: eaa42482c8d046f5bea4db8e85534f78, State: Running, Role: LEADER
I20260620 18:44:19.468098 20963 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa42482c8d046f5bea4db8e85534f78" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43105 } }
I20260620 18:44:19.469115 20964 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eaa42482c8d046f5bea4db8e85534f78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa42482c8d046f5bea4db8e85534f78" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43105 } } }
I20260620 18:44:19.469231 20964 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:19.469398 20965 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaa42482c8d046f5bea4db8e85534f78. Latest consensus state: current_term: 1 leader_uuid: "eaa42482c8d046f5bea4db8e85534f78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa42482c8d046f5bea4db8e85534f78" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43105 } } }
W20260620 18:44:19.469427 20978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260620 18:44:19.469458 20965 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78 [sys.catalog]: This master's current role is: LEADER
W20260620 18:44:19.469473 20978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:44:19.469529 20982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:19.469726 20982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:19.471171 20982 catalog_manager.cc:1347] Generated new cluster ID: 3762fd47b4dd41cd9d4ee9d342626b16
I20260620 18:44:19.471228 20982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:44:19.474876 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.468270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.15.77.126:43105)
0620 18:44:19.469444 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.469454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.469487 (+ 33us) server_negotiation.cc:407] Connection header received
0620 18:44:19.469609 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.469616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.469688 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.469869 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:19.470071 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.471657 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.472706 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.473330 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.473465 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.474169 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:19.474174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:19.474174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:19.474393 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:19.474436 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:19.474451 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:19.474522 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:19.474586 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 18:44:19.474667 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1102,"thread_start_us":757,"threads_started":1}
I20260620 18:44:19.475870 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:44:19.496600 20982 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:19.497400 20982 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:19.503482 20982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eaa42482c8d046f5bea4db8e85534f78: Generated new TSK 0
I20260620 18:44:19.503695 20982 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 18:44:19.527567 20990 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260620 18:44:19.527669 20990 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 18:44:19.527988 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.513506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55224 (local address 127.15.77.126:43105)
0620 18:44:19.513551 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.513553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.513627 (+ 74us) server_negotiation.cc:407] Connection header received
0620 18:44:19.513720 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.513721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.513731 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.513792 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0620 18:44:19.513953 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.514430 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.515636 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.515827 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.515880 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.516065 (+ 185us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 18:44:19.527789 (+ 11724us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 18:44:19.527933 (+ 144us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55224: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 18:44:19.528062 20979 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55224: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260620 18:44:19.528182 20989 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:44:19.513394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:43105 (local address 127.0.0.1:55224)
0620 18:44:19.513563 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:19.513575 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:19.513649 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:19.513816 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:19.513817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:19.513819 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 18:44:19.513878 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:19.513880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.514545 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.514546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:19.515556 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:19.515559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.515907 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.515908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:19.515968 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.515974 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 18:44:19.527924 (+ 11950us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 18:44:19.528067 (+ 143us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.77.126:43105: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43715/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260620 18:44:19.528505 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 20893
2026-06-20T18:44:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (242 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260620 18:44:19.566025 15669 mini_oidc.cc:152] Starting JWKS server
I20260620 18:44:19.566699 15669 webserver.cc:460] Webserver started at https://127.0.0.1:42771/ using document root <none> and password file <none>
I20260620 18:44:19.566780 15669 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 18:44:19.566956 15669 webserver.cc:460] Webserver started at http://127.0.0.1:34637/ using document root <none> and password file <none>
2026-06-20T18:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:19Z Disabled control of system clock
I20260620 18:44:19.574700 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34891
--webserver_interface=127.15.77.126
--webserver_port=0
--builtin_ntp_servers=127.15.77.84:45205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:34891
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42771/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 18:44:19.655205 20996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:19.655436 20996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:19.655459 20996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:19.656661 20996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:19.656704 20996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:19.656718 20996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:19.656730 20996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:19.656742 20996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 18:44:19.656754 20996 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 18:44:19.656766 20996 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42771/jwks.json
I20260620 18:44:19.658133 20996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:34891
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42771/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:19.658396 20996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:19.658699 20996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:19.664049 21002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:19.664016 21001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:19.664081 20996 server_base.cc:1034] running on GCE node
W20260620 18:44:19.664034 21004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:19.664515 20996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:19.664865 20996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:19.666049 20996 hybrid_clock.cc:648] HybridClock initialized: now 1781981059666031 us; error 48 us; skew 500 ppm
I20260620 18:44:19.667447 20996 webserver.cc:460] Webserver started at http://127.15.77.126:38165/ using document root <none> and password file <none>
I20260620 18:44:19.667707 20996 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:19.667779 20996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:19.667937 20996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:19.669101 20996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "47742305efda41eb84ab59b3a3d1eedf"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "c28fc80846743df0fe9d30bc2c94bc77"
server_key_iv: "c4a97131a3388d4aae246bf2c8fb94b2"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.669483 20996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "47742305efda41eb84ab59b3a3d1eedf"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "c28fc80846743df0fe9d30bc2c94bc77"
server_key_iv: "c4a97131a3388d4aae246bf2c8fb94b2"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.670801 20996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:19.671614 21010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:19.671797 20996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:19.671902 20996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "47742305efda41eb84ab59b3a3d1eedf"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "c28fc80846743df0fe9d30bc2c94bc77"
server_key_iv: "c4a97131a3388d4aae246bf2c8fb94b2"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.671988 20996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:19.685986 20996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:19.686345 20996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:19.686501 20996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:19.690536 20996 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:34891
I20260620 18:44:19.690572 21062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:34891 every 8 connection(s)
I20260620 18:44:19.690922 20996 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:19.691648 21063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:19.694418 21063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf: Bootstrap starting.
I20260620 18:44:19.695011 21063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:19.695302 21063 log.cc:824] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:19.696172 21063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf: No bootstrap required, opened a new log
I20260620 18:44:19.697769 21063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47742305efda41eb84ab59b3a3d1eedf" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34891 } }
I20260620 18:44:19.697912 21063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:19.697935 21063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47742305efda41eb84ab59b3a3d1eedf, State: Initialized, Role: FOLLOWER
I20260620 18:44:19.698143 21063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47742305efda41eb84ab59b3a3d1eedf" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34891 } }
I20260620 18:44:19.698226 21063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:19.698274 21063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:19.698323 21063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:19.698727 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 20996
I20260620 18:44:19.698845 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:19.698904 21063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47742305efda41eb84ab59b3a3d1eedf" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34891 } }
I20260620 18:44:19.699025 21063 leader_election.cc:302] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47742305efda41eb84ab59b3a3d1eedf; no voters:
I20260620 18:44:19.699075 15669 external_mini_cluster.cc:1428] Setting key e8a5e2226c5e17dad4b71a9606be965d
I20260620 18:44:19.699222 21063 leader_election.cc:288] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:19.699251 21066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:19.699388 21066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [term 1 LEADER]: Becoming Leader. State: Replica: 47742305efda41eb84ab59b3a3d1eedf, State: Running, Role: LEADER
I20260620 18:44:19.699493 21066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47742305efda41eb84ab59b3a3d1eedf" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34891 } }
I20260620 18:44:19.699548 21063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:19.699815 21066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47742305efda41eb84ab59b3a3d1eedf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47742305efda41eb84ab59b3a3d1eedf" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34891 } } }
I20260620 18:44:19.699887 21066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:19.699910 21067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47742305efda41eb84ab59b3a3d1eedf. Latest consensus state: current_term: 1 leader_uuid: "47742305efda41eb84ab59b3a3d1eedf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47742305efda41eb84ab59b3a3d1eedf" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34891 } } }
I20260620 18:44:19.700007 21067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:19.700340 21072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:19.700904 21072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:19.702461 21072 catalog_manager.cc:1347] Generated new cluster ID: 074f4df034094ebcafc97835b8790fef
I20260620 18:44:19.702518 21072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:44:19.705768 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.699596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36330 (local address 127.15.77.126:34891)
0620 18:44:19.699979 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.699990 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.700099 (+ 109us) server_negotiation.cc:407] Connection header received
0620 18:44:19.700225 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.700233 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.700309 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.700795 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:19.700903 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.702429 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.703542 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.704156 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.704281 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.705046 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:19.705051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:19.705051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:19.705253 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:19.705297 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:19.705311 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:19.705378 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:19.705480 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 18:44:19.705556 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":86,"threads_started":1}
I20260620 18:44:19.706698 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:44:19.709931 21072 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:19.710459 21072 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:19.717039 21072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47742305efda41eb84ab59b3a3d1eedf: Generated new TSK 0
I20260620 18:44:19.717243 21072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:44:19.729004 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.724599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.15.77.126:34891)
0620 18:44:19.724673 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.724676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.724750 (+ 74us) server_negotiation.cc:407] Connection header received
0620 18:44:19.724829 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.724830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.724843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.724930 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0620 18:44:19.725057 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.725864 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.726658 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.727457 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.727532 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.727733 (+ 201us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 18:44:19.728774 (+ 1041us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42771/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 18:44:19.728919 (+ 145us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36338: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42771/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":47}
W20260620 18:44:19.729072 21070 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36338: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42771/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260620 18:44:19.731035 21092 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:44:19.724507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:34891 (local address 127.0.0.1:36338)
0620 18:44:19.724686 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:19.724697 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:19.724765 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:19.724938 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:19.724939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:19.724941 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 18:44:19.724994 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:19.724996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.725970 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.725971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:19.726561 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:19.726562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.727545 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.727546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:19.727596 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.727601 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 18:44:19.728892 (+ 1291us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42771/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 18:44:19.730934 (+ 2042us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.77.126:34891: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42771/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":1833,"thread_start_us":127,"threads_started":1}
I20260620 18:44:19.731582 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 20996
E20260620 18:44:19.740418 21093 webserver.cc:533] Webserver: sslize failed:
E20260620 18:44:19.740494 21093 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-20T18:44:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (203 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-20T18:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:19Z Disabled control of system clock
I20260620 18:44:19.777338 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:35563
--webserver_interface=127.15.77.126
--webserver_port=0
--builtin_ntp_servers=127.15.77.84:38943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:35563
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/testchainca.cert with env {}
W20260620 18:44:19.859617 21097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:19.859879 21097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:19.859905 21097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:19.861174 21097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:19.861227 21097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:19.861238 21097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:19.861253 21097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:19.861265 21097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 18:44:19.861325 21097 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 18:44:19.861342 21097 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260620 18:44:19.862942 21097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:35563
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:35563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:19.863207 21097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:19.863502 21097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:19.868589 21103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:19.868608 21102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:19.868592 21105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:19.868867 21097 server_base.cc:1034] running on GCE node
I20260620 18:44:19.869122 21097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:19.869426 21097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:19.870594 21097 hybrid_clock.cc:648] HybridClock initialized: now 1781981059870571 us; error 54 us; skew 500 ppm
I20260620 18:44:19.871896 21097 webserver.cc:460] Webserver started at http://127.15.77.126:33769/ using document root <none> and password file <none>
I20260620 18:44:19.872171 21097 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:19.872241 21097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:19.872391 21097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:19.873481 21097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "beb1507d9ac943eaa3b53d8fa7577062"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "74f2d213380b7f39b3089701b6f9546a"
server_key_iv: "4353ef590ea3778fc1915707e7a115cd"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.873857 21097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "beb1507d9ac943eaa3b53d8fa7577062"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "74f2d213380b7f39b3089701b6f9546a"
server_key_iv: "4353ef590ea3778fc1915707e7a115cd"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.875173 21097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 18:44:19.875969 21111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:19.876150 21097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:19.876255 21097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "beb1507d9ac943eaa3b53d8fa7577062"
format_stamp: "Formatted at 2026-06-20 18:44:19 on dist-test-slave-tvl9"
server_key: "74f2d213380b7f39b3089701b6f9546a"
server_key_iv: "4353ef590ea3778fc1915707e7a115cd"
server_key_version: "encryptionkey@0"
I20260620 18:44:19.876340 21097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:19.887104 21097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:19.887532 21097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:19.887702 21097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:19.891572 21097 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:35563
I20260620 18:44:19.891650 21163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:35563 every 8 connection(s)
I20260620 18:44:19.892030 21097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:19.893061 21164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:19.895517 21164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062: Bootstrap starting.
I20260620 18:44:19.896130 21164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:19.896435 21164 log.cc:824] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:19.897174 21164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062: No bootstrap required, opened a new log
I20260620 18:44:19.898852 21164 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1507d9ac943eaa3b53d8fa7577062" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35563 } }
I20260620 18:44:19.899019 21164 raft_consensus.cc:374] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:19.899065 21164 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb1507d9ac943eaa3b53d8fa7577062, State: Initialized, Role: FOLLOWER
I20260620 18:44:19.899260 21164 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1507d9ac943eaa3b53d8fa7577062" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35563 } }
I20260620 18:44:19.899397 21164 raft_consensus.cc:388] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:19.899464 21164 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:19.899519 21164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:19.900063 21164 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1507d9ac943eaa3b53d8fa7577062" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35563 } }
I20260620 18:44:19.900184 21164 leader_election.cc:302] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beb1507d9ac943eaa3b53d8fa7577062; no voters:
I20260620 18:44:19.900372 21164 leader_election.cc:288] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:19.900419 21167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:19.900616 21167 raft_consensus.cc:686] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [term 1 LEADER]: Becoming Leader. State: Replica: beb1507d9ac943eaa3b53d8fa7577062, State: Running, Role: LEADER
I20260620 18:44:19.900642 21164 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:19.900729 21167 consensus_queue.cc:237] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1507d9ac943eaa3b53d8fa7577062" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35563 } }
I20260620 18:44:19.901134 21168 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beb1507d9ac943eaa3b53d8fa7577062" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1507d9ac943eaa3b53d8fa7577062" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35563 } } }
I20260620 18:44:19.901237 21168 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:19.901150 21169 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [sys.catalog]: SysCatalogTable state changed. Reason: New leader beb1507d9ac943eaa3b53d8fa7577062. Latest consensus state: current_term: 1 leader_uuid: "beb1507d9ac943eaa3b53d8fa7577062" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1507d9ac943eaa3b53d8fa7577062" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 35563 } } }
I20260620 18:44:19.901367 21169 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:19.901463 21171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:19.902019 21171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:19.902025 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 21097
I20260620 18:44:19.902119 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:19.902298 15669 external_mini_cluster.cc:1428] Setting key 5ed8f839122155139922bd2b9cd37e40
I20260620 18:44:19.904829 21171 catalog_manager.cc:1347] Generated new cluster ID: 5bcf82d2265844eb8e7715e2dcc426be
I20260620 18:44:19.904893 21171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:44:19.908646 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.902748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32900 (local address 127.15.77.126:35563)
0620 18:44:19.902916 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.902926 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.904010 (+ 1084us) server_negotiation.cc:407] Connection header received
0620 18:44:19.904168 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.904174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.904255 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.904398 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:19.904640 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.905697 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.906686 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.907145 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.907279 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.908025 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:19.908029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:19.908029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:44:19.908207 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:19.908242 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:44:19.908253 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:19.908324 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:19.908398 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 18:44:19.908477 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
I20260620 18:44:19.909778 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:44:19.920877 21171 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:19.921437 21171 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:19.926103 21171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beb1507d9ac943eaa3b53d8fa7577062: Generated new TSK 0
I20260620 18:44:19.926343 21171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:44:20.001761 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:19.936069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32912 (local address 127.15.77.126:35563)
0620 18:44:19.936140 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:19.936142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:19.936289 (+ 147us) server_negotiation.cc:407] Connection header received
0620 18:44:19.936366 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:19.936367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:19.936377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:19.936442 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0620 18:44:19.936592 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.937076 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.938308 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.938618 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.938659 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.938908 (+ 249us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 18:44:20.001445 (+ 62537us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 18:44:20.001598 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32912: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 18:44:20.001871 21186 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32912: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260620 18:44:20.001863 21193 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:44:19.936014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:35563 (local address 127.0.0.1:32912)
0620 18:44:19.936212 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:19.936224 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:19.936295 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:19.936456 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:19.936457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:19.936459 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 18:44:19.936526 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:19.936528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.937156 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.937157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:19.938158 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:19.938162 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:19.938737 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:19.938738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:19.938800 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:19.938805 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 18:44:20.001587 (+ 62782us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 18:44:20.001742 (+ 155us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.77.126:35563: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260620 18:44:20.002270 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 21097
2026-06-20T18:44:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (250 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:44:20 dist-test-slave-tvl9 krb5kdc[21205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:44:20 dist-test-slave-tvl9 krb5kdc[21205](info): set up 2 sockets
Jun 20 18:44:20 dist-test-slave-tvl9 krb5kdc[21205](info): commencing operation
krb5kdc: starting...
W20260620 18:44:21.228555 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:44:21 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:44:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:44:21.350685 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40443
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:21.433007 21218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:21.433240 21218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:21.433264 21218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:21.434468 21218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:21.434512 21218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:21.434527 21218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:21.434540 21218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:21.434552 21218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:21.436070 21218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:21.436342 21218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:21.436573 21218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:21.441690 21223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:21.441691 21226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:21.441844 21218 server_base.cc:1034] running on GCE node
W20260620 18:44:21.441982 21224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:21.442257 21218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:21.442564 21218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:21.443727 21218 hybrid_clock.cc:648] HybridClock initialized: now 1781981061443717 us; error 33 us; skew 500 ppm
Jun 20 18:44:21 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:21.450502 21218 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:44:21.450990 21218 webserver.cc:460] Webserver started at http://127.15.77.126:44527/ using document root <none> and password file <none>
I20260620 18:44:21.451191 21218 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:21.451232 21218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:21.451381 21218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:21.452430 21218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "7562f2359b294a34b7e0aecddc5b4e0e"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "66b585a54d9d2094008c40e5e758f815"
server_key_iv: "622e4adac76eafbcbccdf87e667355ae"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.452736 21218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "7562f2359b294a34b7e0aecddc5b4e0e"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "66b585a54d9d2094008c40e5e758f815"
server_key_iv: "622e4adac76eafbcbccdf87e667355ae"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.454036 21218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:44:21.454766 21233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:21.454942 21218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:21.455042 21218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "7562f2359b294a34b7e0aecddc5b4e0e"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "66b585a54d9d2094008c40e5e758f815"
server_key_iv: "622e4adac76eafbcbccdf87e667355ae"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.455117 21218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:21.475935 21218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:21.478412 21218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:21.478533 21218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:21.482578 21218 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:40443
I20260620 18:44:21.482613 21285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:40443 every 8 connection(s)
I20260620 18:44:21.483018 21218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:21.483965 21286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:21.485536 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 21218
I20260620 18:44:21.485617 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:21.485821 15669 external_mini_cluster.cc:1428] Setting key 4c9faf8f67b70abe2aa66acfcd72d23f
I20260620 18:44:21.486797 21286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e: Bootstrap starting.
I20260620 18:44:21.487581 21286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:21.487912 21286 log.cc:824] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:21.488757 21286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e: No bootstrap required, opened a new log
I20260620 18:44:21.491101 21286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7562f2359b294a34b7e0aecddc5b4e0e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40443 } }
I20260620 18:44:21.491297 21286 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:21.491375 21286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7562f2359b294a34b7e0aecddc5b4e0e, State: Initialized, Role: FOLLOWER
I20260620 18:44:21.491533 21286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7562f2359b294a34b7e0aecddc5b4e0e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40443 } }
I20260620 18:44:21.491629 21286 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:21.491663 21286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:21.491712 21286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:21.492319 21286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7562f2359b294a34b7e0aecddc5b4e0e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40443 } }
I20260620 18:44:21.492408 21286 leader_election.cc:302] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7562f2359b294a34b7e0aecddc5b4e0e; no voters:
I20260620 18:44:21.492601 21286 leader_election.cc:288] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:21.492656 21291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:21.492799 21291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [term 1 LEADER]: Becoming Leader. State: Replica: 7562f2359b294a34b7e0aecddc5b4e0e, State: Running, Role: LEADER
I20260620 18:44:21.492888 21286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:21.492893 21291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7562f2359b294a34b7e0aecddc5b4e0e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40443 } }
I20260620 18:44:21.493230 21291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7562f2359b294a34b7e0aecddc5b4e0e. Latest consensus state: current_term: 1 leader_uuid: "7562f2359b294a34b7e0aecddc5b4e0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7562f2359b294a34b7e0aecddc5b4e0e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40443 } } }
I20260620 18:44:21.493295 21291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [sys.catalog]: This master's current role is: LEADER
Jun 20 18:44:21 dist-test-slave-tvl9 krb5kdc[21205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:21.494030 21293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7562f2359b294a34b7e0aecddc5b4e0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7562f2359b294a34b7e0aecddc5b4e0e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40443 } } }
I20260620 18:44:21.494118 21293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:21.494261 21300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:21.494817 21300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:21.496613 21300 catalog_manager.cc:1347] Generated new cluster ID: d083853d1812469d9151d8cfeefc7c74
I20260620 18:44:21.496658 21300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:44:21.497800 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:21.486384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47898 (local address 127.15.77.126:40443)
0620 18:44:21.486590 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:21.486600 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:21.486642 (+ 42us) server_negotiation.cc:407] Connection header received
0620 18:44:21.487231 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:21.487242 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:21.487338 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:21.487493 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:21.488292 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:21.489692 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:21.490450 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:21.491161 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:21.491279 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:21.493895 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:21.493900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:21.493900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:21.493912 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:21.496294 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:21.496657 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:21.496660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:21.496661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:21.496757 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:21.496978 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:21.496979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:21.496979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:21.497221 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:21.497361 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:21.497484 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 18:44:21.497626 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260620 18:44:21.516355 21300 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:21.517091 21300 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:21.530754 21300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7562f2359b294a34b7e0aecddc5b4e0e: Generated new TSK 0
I20260620 18:44:21.530988 21300 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:44:21.566538 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40443
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:21.653355 21312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:21.653573 21312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:21.653599 21312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:21.654800 21312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:21.654844 21312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:21.654872 21312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:44:21.656471 21312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:21.656726 21312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:21.656987 21312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:21.662369 21317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:21.662348 21320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:21.662350 21318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:21.662616 21312 server_base.cc:1034] running on GCE node
I20260620 18:44:21.662817 21312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:21.663025 21312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:21.664171 21312 hybrid_clock.cc:648] HybridClock initialized: now 1781981061664159 us; error 29 us; skew 500 ppm
Jun 20 18:44:21 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:21.666684 21312 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:44:21.667075 21312 webserver.cc:460] Webserver started at http://127.15.77.65:38065/ using document root <none> and password file <none>
I20260620 18:44:21.667277 21312 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:21.667366 21312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:21.667549 21312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:21.668565 21312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "3ffc0c1bf56042ebbc15b35bd3539845"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "cd5dcd2ce9ad947fe6e05b569de165ef"
server_key_iv: "2fd1b06048046855bf9c00455879cf93"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.668872 21312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "3ffc0c1bf56042ebbc15b35bd3539845"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "cd5dcd2ce9ad947fe6e05b569de165ef"
server_key_iv: "2fd1b06048046855bf9c00455879cf93"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.670136 21312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:21.670819 21327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:21.671077 21312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:21.671157 21312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "3ffc0c1bf56042ebbc15b35bd3539845"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "cd5dcd2ce9ad947fe6e05b569de165ef"
server_key_iv: "2fd1b06048046855bf9c00455879cf93"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.671216 21312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:21.691691 21312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:21.694169 21312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:21.694293 21312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:21.694542 21312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:21.694924 21312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:21.694976 21312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:21.695024 21312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:21.695050 21312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:21.700718 21312 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:34161
I20260620 18:44:21.700783 21440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:34161 every 8 connection(s)
I20260620 18:44:21.701116 21312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:44:21.701901 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 21312
I20260620 18:44:21.702037 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:44:21.702287 15669 external_mini_cluster.cc:1428] Setting key e777e706c387be55ccca717cb7cb4fc5
Jun 20 18:44:21 dist-test-slave-tvl9 krb5kdc[21205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:44:21.711702 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:21.701982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:45177 (local address 127.15.77.126:40443)
0620 18:44:21.702045 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:21.702047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:21.702295 (+ 248us) server_negotiation.cc:407] Connection header received
0620 18:44:21.703437 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:21.703438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:21.703452 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:21.703526 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:21.704142 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:21.704751 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:21.705948 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:21.706148 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:21.706227 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:21.708708 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:21.708709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:21.708710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:21.708718 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:21.710164 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:21.710597 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:21.710599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:21.710599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:21.710666 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:21.710954 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:21.710954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:21.710955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:21.711075 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:21.711153 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:21.711500 (+ 347us) server_negotiation.cc:299] Negotiation successful
0620 18:44:21.711596 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 18:44:21.711905 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:21.701955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40443 (local address 127.15.77.65:45177)
0620 18:44:21.702143 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:21.702187 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:21.703300 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:21.703591 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:21.703595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:21.703847 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:21.704038 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:21.704042 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:21.704857 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:21.704858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:21.705870 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:21.705873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:21.706251 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:21.706252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:21.706384 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:21.706916 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:21.706925 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:21.708604 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:21.710268 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:21.710276 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:21.710279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:21.710511 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:21.710735 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:21.710736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:21.710736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:21.710876 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:21.711149 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:21.711158 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:21.711305 (+ 147us) client_negotiation.cc:769] Sending connection context
0620 18:44:21.711477 (+ 172us) client_negotiation.cc:240] Negotiation successful
0620 18:44:21.711648 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
I20260620 18:44:21.712433 21441 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40443
I20260620 18:44:21.712517 21441 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:21.712728 21441 heartbeater.cc:511] Master 127.15.77.126:40443 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
I20260620 18:44:21.713361 21250 ts_manager.cc:194] Registered new tserver with Master: 3ffc0c1bf56042ebbc15b35bd3539845 (127.15.77.65:34161)
I20260620 18:44:21.713977 21250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:45177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:44:21.727933 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40443
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:21.812887 21446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:21.813107 21446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:21.813133 21446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:21.814329 21446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:21.814376 21446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:21.814402 21446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:44:21.815953 21446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:21.816236 21446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:21.816475 21446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:21.821700 21454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:21.821700 21451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:21.821933 21446 server_base.cc:1034] running on GCE node
W20260620 18:44:21.821966 21452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:21.822252 21446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:21.822503 21446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:21.823643 21446 hybrid_clock.cc:648] HybridClock initialized: now 1781981061823629 us; error 30 us; skew 500 ppm
Jun 20 18:44:21 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:21.826324 21446 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:44:21.826766 21446 webserver.cc:460] Webserver started at http://127.15.77.66:36993/ using document root <none> and password file <none>
I20260620 18:44:21.827004 21446 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:21.827073 21446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:21.827230 21446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:21.828326 21446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "58cf14adde4c48d59a794f31ad33e552"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "ae9434b76811400b0494bcda9495f54a"
server_key_iv: "99c44eca07901596aefbfd0c8f78fd29"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.828696 21446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "58cf14adde4c48d59a794f31ad33e552"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "ae9434b76811400b0494bcda9495f54a"
server_key_iv: "99c44eca07901596aefbfd0c8f78fd29"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.830049 21446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:44:21.830852 21461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:21.831040 21446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:21.831143 21446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "58cf14adde4c48d59a794f31ad33e552"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "ae9434b76811400b0494bcda9495f54a"
server_key_iv: "99c44eca07901596aefbfd0c8f78fd29"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.831228 21446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:21.863035 21446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:21.865257 21446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:21.865438 21446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:21.865708 21446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:21.866044 21446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:21.866110 21446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:21.866155 21446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:21.866189 21446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:21.871976 21446 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:41861
I20260620 18:44:21.872026 21574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:41861 every 8 connection(s)
I20260620 18:44:21.872438 21446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:44:21.873127 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 21446
I20260620 18:44:21.873236 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:44:21.873447 15669 external_mini_cluster.cc:1428] Setting key 84be1e9d423b6a212ebe96f0bebfdf60
Jun 20 18:44:21 dist-test-slave-tvl9 krb5kdc[21205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
I20260620 18:44:21.883059 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:21.873288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:59353 (local address 127.15.77.126:40443)
0620 18:44:21.873343 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:21.873345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:21.873562 (+ 217us) server_negotiation.cc:407] Connection header received
0620 18:44:21.874760 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:21.874761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:21.874774 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:21.874853 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:21.875591 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:21.876100 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:21.877135 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:21.877399 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:21.877472 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:21.880053 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:21.880055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:21.880055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:21.880063 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:21.881513 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:21.882027 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:21.882029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:21.882029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:21.882097 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:21.882358 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:21.882359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:21.882360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:21.882485 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:21.882564 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:21.882869 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 18:44:21.882956 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 18:44:21.883267 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:21.873269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40443 (local address 127.15.77.66:59353)
0620 18:44:21.873426 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:21.873465 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:21.874646 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:21.874927 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:21.874932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:21.875231 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:21.875490 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:21.875495 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:21.876203 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:21.876204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:21.877021 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:21.877023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:21.877513 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:21.877514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:21.877690 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:21.878251 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:21.878263 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:21.879950 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:21.881647 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:21.881655 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:21.881657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:21.881950 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:21.882165 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:21.882166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:21.882167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:21.882289 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:21.882554 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:21.882561 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:21.882725 (+ 164us) client_negotiation.cc:769] Sending connection context
0620 18:44:21.882837 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 18:44:21.883004 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260620 18:44:21.883816 21575 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40443
I20260620 18:44:21.883977 21575 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
I20260620 18:44:21.884208 21575 heartbeater.cc:511] Master 127.15.77.126:40443 requested a full tablet report, sending...
I20260620 18:44:21.884680 21250 ts_manager.cc:194] Registered new tserver with Master: 58cf14adde4c48d59a794f31ad33e552 (127.15.77.66:41861)
I20260620 18:44:21.885221 21250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:59353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:44:21.898458 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40443
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:21.980755 21580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:21.980984 21580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:21.981010 21580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:21.982227 21580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:21.982271 21580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:21.982328 21580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:44:21.983949 21580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:21.984217 21580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:21.984493 21580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:21.989867 21588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:21.989867 21585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:21.989898 21586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:21.989979 21580 server_base.cc:1034] running on GCE node
I20260620 18:44:21.990304 21580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:21.990574 21580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:21.991750 21580 hybrid_clock.cc:648] HybridClock initialized: now 1781981061991738 us; error 35 us; skew 500 ppm
Jun 20 18:44:21 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:21.994621 21580 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:44:21.995144 21580 webserver.cc:460] Webserver started at http://127.15.77.67:44563/ using document root <none> and password file <none>
I20260620 18:44:21.995405 21580 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:21.995455 21580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:21.995584 21580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:21.996675 21580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "fb7b55c576904e2a9797ad7d271c26a5"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "c733218e920993e58cd5469398099dc0"
server_key_iv: "e34e78c2a20c3978942fcf4c31d10aaf"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.996985 21580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "fb7b55c576904e2a9797ad7d271c26a5"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "c733218e920993e58cd5469398099dc0"
server_key_iv: "e34e78c2a20c3978942fcf4c31d10aaf"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.998468 21580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:21.999213 21595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:21.999456 21580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:21.999552 21580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "fb7b55c576904e2a9797ad7d271c26a5"
format_stamp: "Formatted at 2026-06-20 18:44:21 on dist-test-slave-tvl9"
server_key: "c733218e920993e58cd5469398099dc0"
server_key_iv: "e34e78c2a20c3978942fcf4c31d10aaf"
server_key_version: "encryptionkey@0"
I20260620 18:44:21.999631 21580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:22.022305 21580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:22.024485 21580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:22.024641 21580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:22.024902 21580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:22.025238 21580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:22.025297 21580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:22.025341 21580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:22.025369 21580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:22.031723 21580 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:38065
I20260620 18:44:22.031761 21708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:38065 every 8 connection(s)
I20260620 18:44:22.032119 21580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:44:22.033432 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 21580
I20260620 18:44:22.033519 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:44:22.033684 15669 external_mini_cluster.cc:1428] Setting key ed190ba4b823b9cfa6ff6cb9b223b7ea
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:44:22.042171 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:22.032932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:37337 (local address 127.15.77.126:40443)
0620 18:44:22.033002 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:22.033004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:22.033257 (+ 253us) server_negotiation.cc:407] Connection header received
0620 18:44:22.034543 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:22.034544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:22.034557 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:22.034627 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:22.035414 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:22.035958 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:22.036960 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:22.037171 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:22.037244 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:22.039563 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:22.039565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:22.039565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:22.039572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:22.040925 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:22.041293 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:22.041294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:22.041294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:22.041345 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:22.041600 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:22.041601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:22.041601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:22.041720 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:22.041785 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:22.042022 (+ 237us) server_negotiation.cc:299] Negotiation successful
0620 18:44:22.042100 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 18:44:22.042418 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:22.032920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40443 (local address 127.15.77.67:37337)
0620 18:44:22.033089 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:22.033128 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:22.034417 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:22.034738 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:22.034742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:22.035071 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:22.035281 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:22.035286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:22.036041 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:22.036042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:22.036867 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:22.036869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:22.037283 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:22.037284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:22.037402 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:22.037941 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:22.037951 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:22.039424 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:22.040998 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:22.041004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:22.041006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:22.041226 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:22.041403 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:22.041404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:22.041404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:22.041503 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:22.041785 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:22.041789 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:22.041906 (+ 117us) client_negotiation.cc:769] Sending connection context
0620 18:44:22.041990 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 18:44:22.042110 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260620 18:44:22.043107 21709 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40443
I20260620 18:44:22.043203 21709 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:22.043408 21709 heartbeater.cc:511] Master 127.15.77.126:40443 requested a full tablet report, sending...
I20260620 18:44:22.043802 21250 ts_manager.cc:194] Registered new tserver with Master: fb7b55c576904e2a9797ad7d271c26a5 (127.15.77.67:38065)
I20260620 18:44:22.044201 21250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:37337
I20260620 18:44:22.045508 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/test-admin.keytab.
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:22.059736 15669 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260620 18:44:22.061450 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:44:22.060608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.15.77.65:34161)
0620 18:44:22.060771 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:22.060776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:22.060840 (+ 64us) server_negotiation.cc:407] Connection header received
0620 18:44:22.060891 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:22.060893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:22.060907 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:22.060974 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:22.061339 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38356: BlockingRecv error: recv got EOF from 127.0.0.1:38356 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
W20260620 18:44:22.061573 15669 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5cc' of type FILE
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:22.062611 15669 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:22.063553 15669 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:44:22.071103 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:22.064072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.15.77.65:34161)
0620 18:44:22.064126 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:22.064128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:22.064180 (+ 52us) server_negotiation.cc:407] Connection header received
0620 18:44:22.064226 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:22.064227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:22.064237 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:22.064283 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:22.064624 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:22.065337 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:22.066034 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:22.066362 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:22.066420 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:22.068634 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:22.068642 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:22.068642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:22.068649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:22.070040 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:22.070365 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:22.070368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:22.070368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:22.070418 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:22.070599 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:22.070600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:22.070600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:22.070725 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:22.070833 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:22.070890 (+ 57us) server_negotiation.cc:299] Negotiation successful
0620 18:44:22.070950 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260620 18:44:22.072944 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:44:22.072418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38378 (local address 127.15.77.65:34161)
0620 18:44:22.072469 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:22.072471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:22.072507 (+ 36us) server_negotiation.cc:407] Connection header received
0620 18:44:22.072593 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:22.072594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:22.072603 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:22.072647 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:22.072889 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38378: BlockingRecv error: recv got EOF from 127.0.0.1:38378 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260620 18:44:22.073076 15669 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5cc' of type FILE
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:22.074061 15669 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:22.074944 15669 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:44:22.081498 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:22.075497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38394 (local address 127.15.77.65:34161)
0620 18:44:22.075534 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:22.075535 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:22.075573 (+ 38us) server_negotiation.cc:407] Connection header received
0620 18:44:22.075623 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:22.075624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:22.075630 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:22.075675 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:22.076002 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:22.076489 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:22.077121 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:22.077306 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:22.077353 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:22.079374 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:22.079376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:22.079376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:22.079382 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:22.080541 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:22.080851 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:22.080852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:22.080852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:22.080898 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:22.081050 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:22.081051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:22.081051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:22.081145 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:22.081214 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:22.081307 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 18:44:22.081363 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260620 18:44:22.083055 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:44:22.082520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38408 (local address 127.15.77.65:34161)
0620 18:44:22.082553 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:22.082555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:22.082612 (+ 57us) server_negotiation.cc:407] Connection header received
0620 18:44:22.082661 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:22.082662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:22.082669 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:22.082716 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:22.082996 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38408: BlockingRecv error: recv got EOF from 127.0.0.1:38408 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 18:44:22.083154 15669 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781981032911509-15669-0/krb5kdc/krb5cc' of type FILE
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:22.084097 15669 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:22.084936 15669 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:44:22.091859 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:22.085393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38412 (local address 127.15.77.65:34161)
0620 18:44:22.085451 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:22.085453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:22.085505 (+ 52us) server_negotiation.cc:407] Connection header received
0620 18:44:22.085598 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:22.085599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:22.085611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:22.085670 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:22.086036 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:22.086492 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:22.087196 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:22.087439 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:22.087510 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:22.089551 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:22.089552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:22.089553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:22.089559 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:22.090796 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:22.091106 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:22.091107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:22.091107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:22.091157 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:22.091421 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:22.091422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:22.091422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:22.091527 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:22.091617 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:22.091734 (+ 117us) server_negotiation.cc:299] Negotiation successful
0620 18:44:22.091794 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 18:44:22.092478 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 21312
I20260620 18:44:22.097363 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 21446
I20260620 18:44:22.102166 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 21580
I20260620 18:44:22.106472 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 21218
2026-06-20T18:44:22Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1923 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21729](info): set up 2 sockets
Jun 20 18:44:22 dist-test-slave-tvl9 krb5kdc[21729](info): commencing operation
krb5kdc: starting...
W20260620 18:44:23.139653 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:44:23 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:44:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.126.keytab.
Entry for principal oryx/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.126.keytab.
I20260620 18:44:23.262244 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:38851
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.126.keytab
--principal=oryx/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:43161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:38851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:23.345149 21742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:23.345418 21742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:23.345460 21742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:23.346777 21742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:23.346853 21742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:23.346876 21742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:23.346896 21742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:23.346927 21742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:23.348481 21742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:43161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:38851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.126.keytab
--principal=oryx/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:38851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:23.348778 21742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:23.349013 21742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:23.353879 21750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:23.354017 21748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:23.354233 21747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:23.354244 21742 server_base.cc:1034] running on GCE node
I20260620 18:44:23.354508 21742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:23.354843 21742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:23.356009 21742 hybrid_clock.cc:648] HybridClock initialized: now 1781981063355978 us; error 69 us; skew 500 ppm
Jun 20 18:44:23 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:23.358501 21742 init.cc:377] Logged in from keytab as oryx/127.15.77.126@KRBTEST.COM (short username oryx)
I20260620 18:44:23.358985 21742 webserver.cc:460] Webserver started at http://127.15.77.126:46319/ using document root <none> and password file <none>
I20260620 18:44:23.359218 21742 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:23.359287 21742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:23.359457 21742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:23.360517 21742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "3afa68609e9b4416a496e310aba3a205"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "5a26b054e987109175422bf0cacd090a"
server_key_iv: "ae0af637cd94feb3776480073fb93643"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.360821 21742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "3afa68609e9b4416a496e310aba3a205"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "5a26b054e987109175422bf0cacd090a"
server_key_iv: "ae0af637cd94feb3776480073fb93643"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.362121 21742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:23.362825 21757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.363018 21742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:23.363106 21742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "3afa68609e9b4416a496e310aba3a205"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "5a26b054e987109175422bf0cacd090a"
server_key_iv: "ae0af637cd94feb3776480073fb93643"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.363163 21742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:23.380133 21742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:23.382025 21742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:23.382145 21742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:23.386271 21742 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:38851
I20260620 18:44:23.386332 21809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:38851 every 8 connection(s)
I20260620 18:44:23.386687 21742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:23.387795 21810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:23.387943 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 21742
I20260620 18:44:23.388055 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:23.388274 15669 external_mini_cluster.cc:1428] Setting key 700c9a7ec3ad3abb5f6801dae0e72320
I20260620 18:44:23.391359 21810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205: Bootstrap starting.
I20260620 18:44:23.392127 21810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:23.392449 21810 log.cc:824] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:23.393204 21810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205: No bootstrap required, opened a new log
Jun 20 18:44:23 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:23.395874 21810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } }
I20260620 18:44:23.396063 21810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:23.396118 21810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afa68609e9b4416a496e310aba3a205, State: Initialized, Role: FOLLOWER
I20260620 18:44:23.396276 21810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } }
I20260620 18:44:23.396404 21810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:23.396452 21810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:23.396514 21810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:23.397075 21810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } }
I20260620 18:44:23.397218 21810 leader_election.cc:302] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3afa68609e9b4416a496e310aba3a205; no voters:
I20260620 18:44:23.397418 21810 leader_election.cc:288] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:23.397449 21815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:23.397555 21815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 1 LEADER]: Becoming Leader. State: Replica: 3afa68609e9b4416a496e310aba3a205, State: Running, Role: LEADER
I20260620 18:44:23.397658 21815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } }
I20260620 18:44:23.397706 21810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:23.397986 21815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } }
I20260620 18:44:23.398059 21815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:23.398185 21815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3afa68609e9b4416a496e310aba3a205. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } }
I20260620 18:44:23.398253 21815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:23.398962 21820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:23.399569 21820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:23.400152 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:23.388870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40090 (local address 127.15.77.126:38851)
0620 18:44:23.389061 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:23.389070 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:23.389107 (+ 37us) server_negotiation.cc:407] Connection header received
0620 18:44:23.389533 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:23.389539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:23.389596 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:23.389714 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:23.390557 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.391614 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.392702 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.393217 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.393361 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:23.396019 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:23.396024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:23.396024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:23.396037 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:23.398171 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:23.398617 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:23.398620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:23.398621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:23.398739 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:23.398967 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:23.398968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:23.398968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:23.399487 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:23.399632 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:23.399794 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 18:44:23.399938 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260620 18:44:23.401117 21820 catalog_manager.cc:1347] Generated new cluster ID: 07d816b82584432a9bbebaff644d5710
I20260620 18:44:23.401177 21820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:44:23.420471 21820 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:23.421026 21820 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:23.426577 21820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205: Generated new TSK 0
I20260620 18:44:23.426815 21820 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab.
Entry for principal oryx/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab.
I20260620 18:44:23.456009 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38851
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab
--principal=oryx/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:43161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:23.551537 21836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:23.551738 21836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:23.551762 21836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:23.552872 21836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:23.552911 21836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:23.552937 21836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:44:23.554323 21836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:43161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab
--principal=oryx/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:23.554519 21836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:23.554705 21836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:23.559715 21841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:23.559849 21842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:23.559718 21844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:23.559937 21836 server_base.cc:1034] running on GCE node
I20260620 18:44:23.560205 21836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:23.560468 21836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:23.561643 21836 hybrid_clock.cc:648] HybridClock initialized: now 1781981063561612 us; error 38 us; skew 500 ppm
Jun 20 18:44:23 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:23.564208 21836 init.cc:377] Logged in from keytab as oryx/127.15.77.65@KRBTEST.COM (short username oryx)
I20260620 18:44:23.564648 21836 webserver.cc:460] Webserver started at http://127.15.77.65:37331/ using document root <none> and password file <none>
I20260620 18:44:23.564879 21836 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:23.564949 21836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:23.565104 21836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:23.566287 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "c37364286d744b818ba2ce19ac421134"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "a08b8424bca1d59b36027d1f35f11d0d"
server_key_iv: "9506c25d40d80feb56d2d638c7a1c694"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.566669 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "c37364286d744b818ba2ce19ac421134"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "a08b8424bca1d59b36027d1f35f11d0d"
server_key_iv: "9506c25d40d80feb56d2d638c7a1c694"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.568229 21836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:23.569064 21851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.569264 21836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:23.569368 21836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "c37364286d744b818ba2ce19ac421134"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "a08b8424bca1d59b36027d1f35f11d0d"
server_key_iv: "9506c25d40d80feb56d2d638c7a1c694"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.569454 21836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:23.589322 21836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:23.591560 21836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:23.591755 21836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:23.592016 21836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:23.592350 21836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:23.592407 21836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.592447 21836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:23.592475 21836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.598656 21836 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:41145
I20260620 18:44:23.598697 21964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:41145 every 8 connection(s)
I20260620 18:44:23.599082 21836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:44:23.601751 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 21836
I20260620 18:44:23.601864 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:44:23.602042 15669 external_mini_cluster.cc:1428] Setting key 8aa1ae0e968bffb11c2857351fdb3727
Jun 20 18:44:23 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:23.609292 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:23.599919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:54629 (local address 127.15.77.126:38851)
0620 18:44:23.599986 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:23.599988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:23.600160 (+ 172us) server_negotiation.cc:407] Connection header received
0620 18:44:23.600644 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:23.600645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:23.600655 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:23.600712 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:23.601741 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.602402 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.603474 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.603716 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.603795 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:23.606249 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:23.606251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:23.606251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:23.606258 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:23.607795 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:23.608305 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:23.608305 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:23.608306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:23.608358 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:23.608633 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:23.608633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:23.608633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:23.608738 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:23.608789 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:23.609152 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 18:44:23.609225 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 18:44:23.609578 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:23.599858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:54629)
0620 18:44:23.600024 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:23.600069 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:23.600557 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:23.600776 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:23.600779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:23.601361 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:23.601616 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:23.601624 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.602490 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.602491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:23.603369 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:23.603372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.603818 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.603819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:23.603953 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:23.604517 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:23.604527 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:23.606138 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:23.607878 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:23.607887 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:23.607889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:23.608219 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:23.608423 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:23.608424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:23.608424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:23.608565 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:23.608802 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:23.608810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:23.608990 (+ 180us) client_negotiation.cc:769] Sending connection context
0620 18:44:23.609132 (+ 142us) client_negotiation.cc:240] Negotiation successful
0620 18:44:23.609307 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260620 18:44:23.610144 21965 heartbeater.cc:347] Connected to a master server at 127.15.77.126:38851
I20260620 18:44:23.610241 21965 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:23.610440 21965 heartbeater.cc:511] Master 127.15.77.126:38851 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:44:23.611006 21774 ts_manager.cc:194] Registered new tserver with Master: c37364286d744b818ba2ce19ac421134 (127.15.77.65:41145)
I20260620 18:44:23.611706 21774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:54629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab.
Entry for principal oryx/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab.
I20260620 18:44:23.626397 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38851
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab
--principal=oryx/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:43161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:23.708119 21970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:23.708329 21970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:23.708359 21970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:23.709451 21970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:23.709491 21970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:23.709517 21970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:44:23.710899 21970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:43161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab
--principal=oryx/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:23.711097 21970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:23.711290 21970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:23.716389 21975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:23.716481 21970 server_base.cc:1034] running on GCE node
W20260620 18:44:23.716367 21976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:23.716367 21978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:23.716822 21970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:23.717031 21970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:23.718219 21970 hybrid_clock.cc:648] HybridClock initialized: now 1781981063718201 us; error 30 us; skew 500 ppm
Jun 20 18:44:23 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:23.720801 21970 init.cc:377] Logged in from keytab as oryx/127.15.77.66@KRBTEST.COM (short username oryx)
I20260620 18:44:23.721330 21970 webserver.cc:460] Webserver started at http://127.15.77.66:35623/ using document root <none> and password file <none>
I20260620 18:44:23.721590 21970 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:23.721733 21970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:23.721916 21970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:23.723017 21970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "1f7bc557402b462d94288962ae3a2be5"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "80790ba1381d5dc80093095e58be1fc6"
server_key_iv: "dc14fb9e69dfe4bda8eda036ac9653e7"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.723443 21970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "1f7bc557402b462d94288962ae3a2be5"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "80790ba1381d5dc80093095e58be1fc6"
server_key_iv: "dc14fb9e69dfe4bda8eda036ac9653e7"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.724822 21970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:23.725572 21985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.725764 21970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:23.725867 21970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "1f7bc557402b462d94288962ae3a2be5"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "80790ba1381d5dc80093095e58be1fc6"
server_key_iv: "dc14fb9e69dfe4bda8eda036ac9653e7"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.725950 21970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:23.736707 21970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:23.738655 21970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:23.738816 21970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:23.739055 21970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:23.739508 21970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:23.739565 21970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.739607 21970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:23.739634 21970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.745522 21970 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:42623
I20260620 18:44:23.745564 22098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:42623 every 8 connection(s)
I20260620 18:44:23.745914 21970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:44:23.751622 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 21970
I20260620 18:44:23.751755 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:44:23.751978 15669 external_mini_cluster.cc:1428] Setting key aa53218b123777e22ab92374729435ec
Jun 20 18:44:23 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:23.756089 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:23.746708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:39629 (local address 127.15.77.126:38851)
0620 18:44:23.746776 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:23.746778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:23.747037 (+ 259us) server_negotiation.cc:407] Connection header received
0620 18:44:23.747581 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:23.747582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:23.747592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:23.747646 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:23.748598 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.749116 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.750094 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.750274 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.750327 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:23.752886 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:23.752888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:23.752888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:23.752895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:23.754416 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:23.754954 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:23.754955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:23.754955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:23.755013 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:23.755304 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:23.755304 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:23.755305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:23.755467 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:23.755524 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:23.755885 (+ 361us) server_negotiation.cc:299] Negotiation successful
0620 18:44:23.755963 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 18:44:23.756487 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:23.746703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:39629)
0620 18:44:23.746900 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:23.746953 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:23.747494 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:23.747705 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:23.747708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:23.748291 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:23.748489 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:23.748494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.749208 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.749209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:23.750012 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:23.750014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.750349 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.750350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:23.750460 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:23.751035 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:23.751044 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:23.752772 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:23.754503 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:23.754511 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:23.754513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:23.754863 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:23.755086 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:23.755087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:23.755088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:23.755236 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:23.755540 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:23.755546 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:23.755738 (+ 192us) client_negotiation.cc:769] Sending connection context
0620 18:44:23.755860 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 18:44:23.756046 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260620 18:44:23.757066 22099 heartbeater.cc:347] Connected to a master server at 127.15.77.126:38851
I20260620 18:44:23.757182 22099 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:23.757401 22099 heartbeater.cc:511] Master 127.15.77.126:38851 requested a full tablet report, sending...
I20260620 18:44:23.757849 21774 ts_manager.cc:194] Registered new tserver with Master: 1f7bc557402b462d94288962ae3a2be5 (127.15.77.66:42623)
I20260620 18:44:23.758232 21774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:39629
WARNING: no policy specified for oryx/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab.
Entry for principal oryx/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab.
I20260620 18:44:23.778059 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38851
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab
--principal=oryx/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:43161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:23.860458 22104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:23.860687 22104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:23.860713 22104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:23.861929 22104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:23.861977 22104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:23.862035 22104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:44:23.863742 22104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:43161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab
--principal=oryx/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:23.864005 22104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:23.864262 22104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:23.869504 22109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:23.869499 22110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:23.869660 22104 server_base.cc:1034] running on GCE node
W20260620 18:44:23.869720 22112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:23.869925 22104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:23.870154 22104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:23.871371 22104 hybrid_clock.cc:648] HybridClock initialized: now 1781981063871347 us; error 33 us; skew 500 ppm
Jun 20 18:44:23 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:23.873970 22104 init.cc:377] Logged in from keytab as oryx/127.15.77.67@KRBTEST.COM (short username oryx)
I20260620 18:44:23.874472 22104 webserver.cc:460] Webserver started at http://127.15.77.67:45521/ using document root <none> and password file <none>
I20260620 18:44:23.874727 22104 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:23.874799 22104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:23.874950 22104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:23.876096 22104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "5e4de4da6a02462b80bdfbde421dc0a6"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "339d51a5879ec070ee2a2d93c514733a"
server_key_iv: "f7f87debde1e36b2d8417a25b1fda94f"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.876469 22104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "5e4de4da6a02462b80bdfbde421dc0a6"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "339d51a5879ec070ee2a2d93c514733a"
server_key_iv: "f7f87debde1e36b2d8417a25b1fda94f"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.877837 22104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:44:23.878614 22119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.878788 22104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.878897 22104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "5e4de4da6a02462b80bdfbde421dc0a6"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "339d51a5879ec070ee2a2d93c514733a"
server_key_iv: "f7f87debde1e36b2d8417a25b1fda94f"
server_key_version: "encryptionkey@0"
I20260620 18:44:23.878974 22104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:23.893225 22104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:23.895824 22104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:23.895999 22104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:23.896255 22104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:23.896586 22104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:23.896646 22104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.896692 22104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:23.896716 22104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:23.903211 22104 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:45993
I20260620 18:44:23.903254 22232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:45993 every 8 connection(s)
I20260620 18:44:23.903652 22104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
Jun 20 18:44:23 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:23.912710 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:23.904429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:57661 (local address 127.15.77.126:38851)
0620 18:44:23.904492 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:23.904494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:23.904738 (+ 244us) server_negotiation.cc:407] Connection header received
0620 18:44:23.905169 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:23.905170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:23.905181 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:23.905236 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:23.906157 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.906654 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.907728 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.907915 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.907969 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:23.909992 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:23.909994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:23.909994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:23.910001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:23.911486 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:23.911834 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:23.911835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:23.911835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:23.911886 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:23.912087 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:23.912088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:23.912088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:23.912192 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:23.912242 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:23.912498 (+ 256us) server_negotiation.cc:299] Negotiation successful
0620 18:44:23.912563 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 18:44:23.912810 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:23.904420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:57661)
0620 18:44:23.904566 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:23.904603 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:23.905081 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:23.905313 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:23.905317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:23.905883 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:23.906080 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:23.906085 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.906731 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.906731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:23.907658 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:23.907660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:23.907985 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:23.907986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:23.908089 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:23.908568 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:23.908576 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:23.909905 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:23.911551 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:23.911557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:23.911559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:23.911772 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:23.911942 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:23.911942 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:23.911942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:23.912036 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:23.912244 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:23.912248 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:23.912365 (+ 117us) client_negotiation.cc:769] Sending connection context
0620 18:44:23.912473 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 18:44:23.912598 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260620 18:44:23.913275 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 22104
I20260620 18:44:23.913372 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:44:23.913355 22233 heartbeater.cc:347] Connected to a master server at 127.15.77.126:38851
I20260620 18:44:23.913460 22233 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:23.913568 15669 external_mini_cluster.cc:1428] Setting key 19b77b8fadb4ea5ac40007b9ef3e5910
I20260620 18:44:23.913676 22233 heartbeater.cc:511] Master 127.15.77.126:38851 requested a full tablet report, sending...
I20260620 18:44:23.914132 21774 ts_manager.cc:194] Registered new tserver with Master: 5e4de4da6a02462b80bdfbde421dc0a6 (127.15.77.67:45993)
I20260620 18:44:23.914520 21774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:57661
I20260620 18:44:23.914986 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.77.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.125.keytab.
Entry for principal oryx/127.15.77.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.125.keytab.
I20260620 18:44:23.939940 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.125:40663
--webserver_interface=127.15.77.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.125.keytab
--principal=oryx/127.15.77.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:43161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:38851,127.15.77.125:40663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:24.021905 22238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:24.022171 22238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:24.022209 22238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:24.023475 22238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:24.023542 22238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:24.023566 22238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:24.023595 22238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:24.023622 22238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:24.025154 22238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:43161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:38851,127.15.77.125:40663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.125.keytab
--principal=oryx/127.15.77.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.125:40663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.77.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
W20260620 18:44:24.025313 22238 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.77.126:38851,127.15.77.125:40663'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260620 18:44:24.025478 22238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:24.025712 22238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:24.030684 22244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:24.030699 22246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:24.030694 22243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.030982 22238 server_base.cc:1034] running on GCE node
I20260620 18:44:24.031225 22238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:24.031538 22238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:24.032688 22238 hybrid_clock.cc:648] HybridClock initialized: now 1781981064032675 us; error 31 us; skew 500 ppm
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:24.035389 22238 init.cc:377] Logged in from keytab as oryx/127.15.77.125@KRBTEST.COM (short username oryx)
I20260620 18:44:24.035864 22238 webserver.cc:460] Webserver started at http://127.15.77.125:33719/ using document root <none> and password file <none>
I20260620 18:44:24.036065 22238 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:24.036106 22238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:24.036206 22238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:24.037256 22238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data/instance:
uuid: "56cb9590fadb4cf68fc41eced963ab04"
format_stamp: "Formatted at 2026-06-20 18:44:24 on dist-test-slave-tvl9"
server_key: "4c6e4b29e96049c003257088e0e41997"
server_key_iv: "c8eee691d007021b7d0fe5f5431caf14"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.037563 22238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal/instance:
uuid: "56cb9590fadb4cf68fc41eced963ab04"
format_stamp: "Formatted at 2026-06-20 18:44:24 on dist-test-slave-tvl9"
server_key: "4c6e4b29e96049c003257088e0e41997"
server_key_iv: "c8eee691d007021b7d0fe5f5431caf14"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.038935 22238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:24.039743 22253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.039923 22238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 18:44:24.040035 22238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal
uuid: "56cb9590fadb4cf68fc41eced963ab04"
format_stamp: "Formatted at 2026-06-20 18:44:24 on dist-test-slave-tvl9"
server_key: "4c6e4b29e96049c003257088e0e41997"
server_key_iv: "c8eee691d007021b7d0fe5f5431caf14"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.040110 22238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:24.052457 22238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:24.054929 22238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:24.055097 22238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:24.059242 22238 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.125:40663
I20260620 18:44:24.059298 22305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.125:40663 every 8 connection(s)
I20260620 18:44:24.059717 22238 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
I20260620 18:44:24.060730 22306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:24.062259 22306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:44:24.064671 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 22238
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.125@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.77.125@KRBTEST.COM
I20260620 18:44:24.073009 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.062946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40098 (local address 127.15.77.126:38851)
0620 18:44:24.063009 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.063011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.063215 (+ 204us) server_negotiation.cc:407] Connection header received
0620 18:44:24.063734 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.063735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.063745 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.063809 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.064461 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.065261 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.066905 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.067206 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.067285 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.069826 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.069827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.069827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.069835 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.071447 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.071951 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.071953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.071953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.072027 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.072289 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.072290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.072290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.072443 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.072519 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.072799 (+ 280us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.072891 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 18:44:24.073174 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.062916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40098)
0620 18:44:24.063078 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.063121 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.063644 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.063861 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.063865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.064113 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.064317 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.064321 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.065389 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.065391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.066821 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.066824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.067306 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.067307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.067476 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.067991 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.068001 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.069732 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.071538 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.071545 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.071547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.071818 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.072098 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.072099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.072100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.072213 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.072521 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.072528 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.072669 (+ 141us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.072778 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.072928 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":3,"thread_start_us":86,"threads_started":1}
I20260620 18:44:24.073837 22306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:44:24.074193 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.065244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.15.77.125:40663)
0620 18:44:24.065407 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.065412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.065565 (+ 153us) server_negotiation.cc:407] Connection header received
0620 18:44:24.065614 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.065616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.065633 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.065771 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.066600 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.067543 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.068547 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.068834 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.068893 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.071114 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.071121 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.071121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.071128 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.072846 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.073304 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.073307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.073308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.073366 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.073539 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.073540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.073540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.073737 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.073905 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.073993 (+ 88us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.074081 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.125@KRBTEST.COM for oryx/127.15.77.125@KRBTEST.COM
I20260620 18:44:24.080389 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.074104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:40663 (local address 127.0.0.1:60708)
0620 18:44:24.074276 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.074286 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.074333 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.074799 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.074800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.074951 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.074998 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.075000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.075692 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.075693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.076169 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.076170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.076502 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.076503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.076561 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.076979 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.076984 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.078099 (+ 1115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.079486 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.079488 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.079488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.079714 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.079889 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.079889 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.079889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.079940 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.080149 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.080149 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.080184 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.080233 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.080306 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260620 18:44:24.080441 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.074155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60708 (local address 127.15.77.125:40663)
0620 18:44:24.074272 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.074274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.074319 (+ 45us) server_negotiation.cc:407] Connection header received
0620 18:44:24.074386 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.074386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.074396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.074757 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.075071 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.075598 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.076246 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.076427 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.076476 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.078181 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.078182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.078182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.078188 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.079415 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.079779 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.079780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.079781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.079829 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.079991 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.079992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.079992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.080092 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.080142 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.080247 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.080351 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":103,"thread_start_us":84,"threads_started":1}
I20260620 18:44:24.081759 22306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: Bootstrap starting.
I20260620 18:44:24.082600 22306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:24.082922 22306 log.cc:824] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:24.083714 22306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: No bootstrap required, opened a new log
I20260620 18:44:24.085103 22306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } }
I20260620 18:44:24.085258 22306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:24.085307 22306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56cb9590fadb4cf68fc41eced963ab04, State: Initialized, Role: FOLLOWER
I20260620 18:44:24.085443 22306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } }
I20260620 18:44:24.085739 22306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:24.085712 22313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } } }
I20260620 18:44:24.085831 22313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 18:44:24.087313 22324 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:44:24.087394 22324 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:44:24.089046 22238 master_runner.cc:428] Detected that this master 56cb9590fadb4cf68fc41eced963ab04 is joining an existing cluster
I20260620 18:44:24.089087 22238 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.77.125:40663
I20260620 18:44:24.089635 22313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:24.089723 22313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } }
I20260620 18:44:24.089916 22313 leader_election.cc:288] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851)
I20260620 18:44:24.090104 21784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56cb9590fadb4cf68fc41eced963ab04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afa68609e9b4416a496e310aba3a205" is_pre_election: true
I20260620 18:44:24.090482 22257 leader_election.cc:302] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 56cb9590fadb4cf68fc41eced963ab04; no voters: 3afa68609e9b4416a496e310aba3a205
I20260620 18:44:24.090637 22313 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:44:24.097218 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.090882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.15.77.126:38851)
0620 18:44:24.090929 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.090931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.091071 (+ 140us) server_negotiation.cc:407] Connection header received
0620 18:44:24.091120 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.091120 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.091132 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.091187 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.091548 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.092114 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.092832 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.093142 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.093218 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.094276 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.094278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.094278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.094285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.095920 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.096368 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.096369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.096369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.096431 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.096617 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.096618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.096618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.096760 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.096833 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.097014 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.097109 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 18:44:24.097239 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.090071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.15.77.125:40663)
0620 18:44:24.090115 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.090118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.090351 (+ 233us) server_negotiation.cc:407] Connection header received
0620 18:44:24.090442 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.090443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.090453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.090480 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.090953 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.091708 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.092763 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.092997 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.093076 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.094219 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.094220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.094221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.094228 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.095946 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.096419 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.096420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.096420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.096465 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.096696 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.096696 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.096696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.096813 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.096851 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.097036 (+ 185us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.097145 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":21}
I20260620 18:44:24.097479 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.090836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40112)
0620 18:44:24.090990 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.091000 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.091064 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.091191 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.091192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.091406 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.091465 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.091467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.092195 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.092196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.092763 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.092764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.093217 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.093217 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.093288 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.093800 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.093804 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.094210 (+ 406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.096025 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.096026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.096027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.096280 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.096486 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.096487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.096487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.096548 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.096838 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.096839 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.096895 (+ 56us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.096971 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.097385 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260620 18:44:24.097612 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.090126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:40663 (local address 127.0.0.1:60720)
0620 18:44:24.090287 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.090299 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.090376 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.090537 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.090538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.090776 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.090862 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.090865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.091794 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.091795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.092674 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.092676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.093091 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.093092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.093152 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.093639 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.093644 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.094122 (+ 478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.096034 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.096035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.096035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.096348 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.096532 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.096532 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.096532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.096626 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.096888 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.096889 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.096927 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.097062 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.097204 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":52,"thread_start_us":91,"threads_started":1}
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.126@KRBTEST.COM for oryx/127.15.77.125@KRBTEST.COM
I20260620 18:44:24.110803 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.100261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60730 (local address 127.15.77.125:40663)
0620 18:44:24.100295 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.100297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.103557 (+ 3260us) server_negotiation.cc:407] Connection header received
0620 18:44:24.103641 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.103642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.103651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.103696 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.104199 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.104647 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.105764 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.105932 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.105983 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.108268 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.108269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.108270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.108276 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.109650 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.110088 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.110089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.110089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.110138 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.110344 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.110344 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.110344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.110439 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.110489 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.110679 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.110738 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 18:44:24.110903 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.100217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:40663 (local address 127.0.0.1:60730)
0620 18:44:24.103435 (+ 3218us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.103486 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.103565 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.103711 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.103714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.104056 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.104119 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.104121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.104723 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.104724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.105682 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.105685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.106006 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.106007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.106091 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.106544 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.106550 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.108182 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.109733 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.109741 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.109744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.110006 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.110206 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.110207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.110207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.110282 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.110499 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.110507 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.110564 (+ 57us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.110656 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.110786 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3187,"thread_start_us":69,"threads_started":1}
I20260620 18:44:24.111577 21773 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.77.125:40663: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04"
member_type: NON_VOTER
last_known_addr {
host: "127.15.77.125"
port: 40663
}
attrs {
promote: true
}
}
dest_uuid: "3afa68609e9b4416a496e310aba3a205"
cas_config_opid_index: -1
I20260620 18:44:24.111913 21773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } }
I20260620 18:44:24.112457 22334 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 56cb9590fadb4cf68fc41eced963ab04 (127.15.77.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } } }
I20260620 18:44:24.112772 22280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:24.113588 22334 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.77.125:40663
I20260620 18:44:24.113631 22280 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Refusing update from remote peer 3afa68609e9b4416a496e310aba3a205: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260620 18:44:24.113874 22238 master.cc:537] Master@127.15.77.125:40663 shutting down...
I20260620 18:44:24.113850 22333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } } }
I20260620 18:44:24.113880 22334 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:24.113924 22333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: This master's current role is: LEADER
W20260620 18:44:24.114763 21760 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260620 18:44:24.115226 21760 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 -> Peer 56cb9590fadb4cf68fc41eced963ab04 (127.15.77.125:40663): Couldn't send request to peer 56cb9590fadb4cf68fc41eced963ab04. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 18:44:24.116753 22238 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:24.116837 22238 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:24.116873 22238 tablet_replica.cc:330] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: stopping tablet replica
I20260620 18:44:24.129761 22238 master.cc:552] Master@127.15.77.125:40663 shutdown complete.
I20260620 18:44:24.129812 22238 master_runner.cc:305] Clearing existing system tablet
I20260620 18:44:24.129963 22238 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:24.130004 22238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:24.130633 22238 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:24.131129 22338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.131242 22238 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:24.131299 22238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.131403 22238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal
uuid: "56cb9590fadb4cf68fc41eced963ab04"
format_stamp: "Formatted at 2026-06-20 18:44:24 on dist-test-slave-tvl9"
server_key: "4c6e4b29e96049c003257088e0e41997"
server_key_iv: "c8eee691d007021b7d0fe5f5431caf14"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.132089 22238 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:24.133459 22238 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 18:44:24.133527 22238 log.cc:1196] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260620 18:44:24.133766 22238 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: Deleting consensus metadata
I20260620 18:44:24.133894 22238 master_runner.cc:315] Copying system tablet from 127.15.77.126:38851
I20260620 18:44:24.134483 22238 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: tablet copy: Beginning tablet copy session from remote peer at address 127.15.77.126:38851
I20260620 18:44:24.140818 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.134923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40118 (local address 127.15.77.126:38851)
0620 18:44:24.134985 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.134987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.135121 (+ 134us) server_negotiation.cc:407] Connection header received
0620 18:44:24.135197 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.135197 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.135209 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.135268 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.135624 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.136129 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.136908 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.137114 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.137199 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.138177 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.138178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.138179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.138186 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.139646 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.140007 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.140008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.140009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.140068 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.140276 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.140277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.140277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.140392 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.140464 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.140646 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.140723 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 18:44:24.140832 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.134865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40118)
0620 18:44:24.135038 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.135049 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.135138 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.135274 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.135275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.135467 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.135522 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.135524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.136219 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.136220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.136777 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.136779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.137201 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.137202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.137275 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.137763 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.137768 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.138059 (+ 291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.139741 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.139742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.139743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.139920 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.140148 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.140149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.140150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.140212 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.140468 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.140469 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.140516 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.140615 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.140724 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":69,"threads_started":1}
I20260620 18:44:24.141160 21794 tablet_copy_service.cc:140] P 3afa68609e9b4416a496e310aba3a205: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 56cb9590fadb4cf68fc41eced963ab04 ({username='oryx', principal='oryx/127.15.77.125@KRBTEST.COM'} at 127.0.0.1:40118)
I20260620 18:44:24.141255 21794 tablet_copy_service.cc:161] P 3afa68609e9b4416a496e310aba3a205: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 56cb9590fadb4cf68fc41eced963ab04 at {username='oryx', principal='oryx/127.15.77.125@KRBTEST.COM'} at 127.0.0.1:40118: session id = 56cb9590fadb4cf68fc41eced963ab04-00000000000000000000000000000000
I20260620 18:44:24.141924 21794 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 18:44:24.142572 22238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 18:44:24.143596 22238 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: tablet copy: Starting download of 0 data blocks...
I20260620 18:44:24.143755 22238 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: tablet copy: Starting download of 1 WAL segments...
I20260620 18:44:24.144704 22238 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 18:44:24.145670 21794 tablet_copy_service.cc:342] P 3afa68609e9b4416a496e310aba3a205: Request end of tablet copy session 56cb9590fadb4cf68fc41eced963ab04-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.77.125@KRBTEST.COM'} at 127.0.0.1:40118
I20260620 18:44:24.145738 21794 tablet_copy_service.cc:433] P 3afa68609e9b4416a496e310aba3a205: ending tablet copy session 56cb9590fadb4cf68fc41eced963ab04-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 56cb9590fadb4cf68fc41eced963ab04
I20260620 18:44:24.148540 22238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:24.149731 22345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:24.149896 22348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:24.150009 22346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.150214 22238 server_base.cc:1034] running on GCE node
I20260620 18:44:24.150309 22238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:24.150445 22238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:24.151568 22238 hybrid_clock.cc:648] HybridClock initialized: now 1781981064151559 us; error 24 us; skew 500 ppm
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:24.152923 22238 init.cc:377] Logged in from keytab as oryx/127.15.77.125@KRBTEST.COM (short username oryx)
I20260620 18:44:24.153214 22238 webserver.cc:460] Webserver started at http://127.15.77.125:42379/ using document root <none> and password file <none>
I20260620 18:44:24.153352 22238 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:24.153391 22238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:24.154029 22238 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:24.154609 22355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.154728 22238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.154775 22238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal
uuid: "56cb9590fadb4cf68fc41eced963ab04"
format_stamp: "Formatted at 2026-06-20 18:44:24 on dist-test-slave-tvl9"
server_key: "4c6e4b29e96049c003257088e0e41997"
server_key_iv: "c8eee691d007021b7d0fe5f5431caf14"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.154836 22238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:24.183641 22238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:24.185266 22238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:24.188452 22238 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.125:40663
I20260620 18:44:24.188884 22238 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
I20260620 18:44:24.188899 22407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.125:40663 every 8 connection(s)
I20260620 18:44:24.189649 22408 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:44:24.190235 22408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: Bootstrap starting.
I20260620 18:44:24.192569 22408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:44:24.192793 22408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: Bootstrap complete.
I20260620 18:44:24.192998 22408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } }
I20260620 18:44:24.193090 22408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 56cb9590fadb4cf68fc41eced963ab04, State: Initialized, Role: LEARNER
I20260620 18:44:24.193141 22408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } }
I20260620 18:44:24.193329 22408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:24.193320 22411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } } }
I20260620 18:44:24.193403 22411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: This master's current role is: LEARNER
I20260620 18:44:24.194808 22422 catalog_manager.cc:1259] Loaded cluster ID: 07d816b82584432a9bbebaff644d5710
I20260620 18:44:24.194880 22422 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: loading cluster ID for follower catalog manager: success
I20260620 18:44:24.195860 22422 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: acquiring CA information for follower catalog manager: success
I20260620 18:44:24.196156 22422 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:44:24.233237 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.226440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.15.77.125:40663)
0620 18:44:24.226619 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.226621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.226632 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:44:24.226661 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.226662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.226674 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.226749 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.227474 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.228407 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.229168 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.229363 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.229419 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.230620 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.230621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.230621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.230628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.232209 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.232527 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.232528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.232529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.232582 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.232793 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.232794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.232795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.232917 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.233010 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.233086 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.233163 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for oryx/127.15.77.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.124.keytab.
Entry for principal oryx/127.15.77.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.124.keytab.
I20260620 18:44:24.259179 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.124:39841
--webserver_interface=127.15.77.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.124.keytab
--principal=oryx/127.15.77.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:43161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:38851,127.15.77.125:40663,127.15.77.124:39841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:24.341627 22426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:24.341892 22426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:24.341929 22426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:24.343113 22426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:24.343199 22426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:24.343220 22426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:24.343240 22426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:24.343267 22426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:24.344841 22426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:43161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:38851,127.15.77.125:40663,127.15.77.124:39841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.124.keytab
--principal=oryx/127.15.77.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.124:39841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.77.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:24.345139 22426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:24.345384 22426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:24.350330 22431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.350447 22426 server_base.cc:1034] running on GCE node
W20260620 18:44:24.350330 22434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:24.350358 22432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.350764 22426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:24.351059 22426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:24.352249 22426 hybrid_clock.cc:648] HybridClock initialized: now 1781981064352236 us; error 53 us; skew 500 ppm
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:24.354859 22426 init.cc:377] Logged in from keytab as oryx/127.15.77.124@KRBTEST.COM (short username oryx)
I20260620 18:44:24.355304 22426 webserver.cc:460] Webserver started at http://127.15.77.124:40031/ using document root <none> and password file <none>
I20260620 18:44:24.355564 22426 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:24.355609 22426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:24.355700 22426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:24.356817 22426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data/instance:
uuid: "5d433ee43d8b4dd58922cabbe3098519"
format_stamp: "Formatted at 2026-06-20 18:44:24 on dist-test-slave-tvl9"
server_key: "12c25dc58f946e7885fa0029e29e0bdb"
server_key_iv: "49f002f9dc58fcb01e28ab385213e197"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.357129 22426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal/instance:
uuid: "5d433ee43d8b4dd58922cabbe3098519"
format_stamp: "Formatted at 2026-06-20 18:44:24 on dist-test-slave-tvl9"
server_key: "12c25dc58f946e7885fa0029e29e0bdb"
server_key_iv: "49f002f9dc58fcb01e28ab385213e197"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.358372 22426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:44:24.359046 22441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.359213 22426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:44:24.359303 22426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal
uuid: "5d433ee43d8b4dd58922cabbe3098519"
format_stamp: "Formatted at 2026-06-20 18:44:24 on dist-test-slave-tvl9"
server_key: "12c25dc58f946e7885fa0029e29e0bdb"
server_key_iv: "49f002f9dc58fcb01e28ab385213e197"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.359416 22426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:24.374214 22426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:24.376504 22426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:24.376704 22426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:24.380972 22426 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.124:39841
I20260620 18:44:24.381008 22493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.124:39841 every 8 connection(s)
I20260620 18:44:24.381443 22426 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
I20260620 18:44:24.382561 22494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:24.384080 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 22426
I20260620 18:44:24.384336 22494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.124@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.77.124@KRBTEST.COM
I20260620 18:44:24.394678 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.385217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40134 (local address 127.15.77.126:38851)
0620 18:44:24.385270 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.385273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.385308 (+ 35us) server_negotiation.cc:407] Connection header received
0620 18:44:24.385612 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.385614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.385625 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.385682 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.386487 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.386991 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.388257 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.388555 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.388631 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.391154 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.391155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.391156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.391164 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.393069 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.393548 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.393549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.393550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.393640 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.393934 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.393935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.393935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.394060 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.394147 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.394491 (+ 344us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.394587 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 18:44:24.394948 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.385030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40134)
0620 18:44:24.385197 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.385241 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.385516 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.385722 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.385725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.386141 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.386386 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.386393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.387099 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.387100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.388161 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.388163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.388662 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.388663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.388793 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.389421 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.389429 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.391047 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.393219 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.393225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.393227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.393468 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.393717 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.393717 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.393718 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.393837 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.394135 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.394142 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.394348 (+ 206us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.394471 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.394637 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":88,"thread_start_us":84,"threads_started":1}
I20260620 18:44:24.395088 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.384731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52522 (local address 127.15.77.124:39841)
0620 18:44:24.384920 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.384931 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.384963 (+ 32us) server_negotiation.cc:407] Connection header received
0620 18:44:24.385503 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.385510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.385531 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.385678 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.386418 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.387690 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.388560 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.388985 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.389060 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.391772 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.391780 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.391780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.391787 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.393721 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.394063 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.394067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.394068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.394129 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.394367 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.394368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.394368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.394508 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.394682 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.394727 (+ 45us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.394951 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260620 18:44:24.395529 22494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.124@KRBTEST.COM for oryx/127.15.77.125@KRBTEST.COM
I20260620 18:44:24.401660 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.395791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:40663 (local address 127.0.0.1:60772)
0620 18:44:24.395843 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.395855 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.395948 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.396081 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.396082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.396195 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.396232 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.396233 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.396826 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.396826 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.397403 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.397405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.397759 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.397760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.397830 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.398280 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.398285 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.399532 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.400866 (+ 1334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.400867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.400867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.401024 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.401194 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.401194 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.401195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.401242 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.401440 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.401441 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.401473 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.401522 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.401591 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 18:44:24.401683 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.395872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60772 (local address 127.15.77.125:40663)
0620 18:44:24.395919 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.395921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.395929 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:44:24.396011 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.396012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.396025 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.396080 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.396273 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.396763 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.397467 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.397676 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.397730 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.399627 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.399629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.399629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.399636 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.400799 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.401088 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.401089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.401090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.401138 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.401292 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.401293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.401293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.401389 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.401439 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.401537 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.401623 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:44:24.402065 22494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39841 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.124@KRBTEST.COM for oryx/127.15.77.124@KRBTEST.COM
I20260620 18:44:24.407960 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.402339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39841 (local address 127.0.0.1:52532)
0620 18:44:24.402390 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.402400 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.402435 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.402536 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.402537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.402649 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.402677 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.402678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.403241 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.403242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.403779 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.403781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.404097 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.404098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.404132 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.404452 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.404456 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.405536 (+ 1080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.407041 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.407041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.407042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.407222 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.407459 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.407460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.407460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.407509 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.407720 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.407721 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.407754 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.407805 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.407878 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 18:44:24.408047 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.402387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.15.77.124:39841)
0620 18:44:24.402421 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.402423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.402430 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:44:24.402482 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.402483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.402490 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.402535 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.402734 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.403176 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.403848 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.404035 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.404088 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.405605 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.405606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.405606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.405612 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.406971 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.407284 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.407285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.407285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.407405 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.407559 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.407559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.407559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.407668 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.407715 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.407820 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.407921 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":16}
I20260620 18:44:24.409317 22494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: Bootstrap starting.
I20260620 18:44:24.410141 22494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:24.410370 22494 log.cc:824] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:24.411078 22494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: No bootstrap required, opened a new log
I20260620 18:44:24.412547 22494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39841 } }
I20260620 18:44:24.412683 22494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:24.412707 22494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d433ee43d8b4dd58922cabbe3098519, State: Initialized, Role: FOLLOWER
I20260620 18:44:24.412837 22494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39841 } }
I20260620 18:44:24.413121 22499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39841 } } }
I20260620 18:44:24.413196 22494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:24.413242 22499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 18:44:24.414896 22510 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:44:24.414959 22510 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:44:24.415215 22499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:24.415306 22499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39841 } }
I20260620 18:44:24.415598 22499 leader_election.cc:288] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851), 56cb9590fadb4cf68fc41eced963ab04 (127.15.77.125:40663)
I20260620 18:44:24.415709 21784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d433ee43d8b4dd58922cabbe3098519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afa68609e9b4416a496e310aba3a205" is_pre_election: true
I20260620 18:44:24.415791 22382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d433ee43d8b4dd58922cabbe3098519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56cb9590fadb4cf68fc41eced963ab04" is_pre_election: true
I20260620 18:44:24.415915 22382 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 LEARNER]: Handling vote request from an unknown peer 5d433ee43d8b4dd58922cabbe3098519
I20260620 18:44:24.415982 22382 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 5d433ee43d8b4dd58922cabbe3098519 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260620 18:44:24.416177 22444 leader_election.cc:302] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d433ee43d8b4dd58922cabbe3098519; no voters: 3afa68609e9b4416a496e310aba3a205, 56cb9590fadb4cf68fc41eced963ab04
I20260620 18:44:24.416391 22499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:24.417315 22499 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:44:24.417750 22426 master_runner.cc:428] Detected that this master 5d433ee43d8b4dd58922cabbe3098519 is joining an existing cluster
I20260620 18:44:24.417802 22426 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.77.124:39841
I20260620 18:44:24.425772 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.419017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52542 (local address 127.15.77.124:39841)
0620 18:44:24.419054 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.419056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.419109 (+ 53us) server_negotiation.cc:407] Connection header received
0620 18:44:24.419202 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.419203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.419212 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.419242 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.419751 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.420497 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.421568 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.421828 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.422250 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.423245 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.423246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.423246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.423252 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.424679 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.425076 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.425076 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.425077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.425124 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.425332 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.425332 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.425333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.425429 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.425477 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.425643 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.425709 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 18:44:24.425944 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.418869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39841 (local address 127.0.0.1:52542)
0620 18:44:24.419046 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.419057 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.419127 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.419298 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.419299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.419567 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.419651 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.419653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.420587 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.420587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.421483 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.421486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.422269 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.422270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.422341 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.422708 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.422713 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.423160 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.424762 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.424763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.424763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.424987 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.425199 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.425201 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.425201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.425269 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.425491 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.425492 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.425538 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.425625 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.425794 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":70,"spinlock_wait_cycles":9600,"thread_start_us":105,"threads_started":1}
I20260620 18:44:24.446944 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 21836
I20260620 18:44:24.452387 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:41145
--local_ip_for_outbound_sockets=127.15.77.65
--tserver_master_addrs=127.15.77.126:38851
--webserver_port=37331
--webserver_interface=127.15.77.65
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab
--principal=oryx/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:43161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.77.126:38851,127.15.77.125:40663
--tserver_master_addrs=127.15.77.126:38851,127.15.77.125:40663,127.15.77.124:39841 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:44:24.455011 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.418952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.15.77.125:40663)
0620 18:44:24.419003 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.419005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.419384 (+ 379us) server_negotiation.cc:407] Connection header received
0620 18:44:24.419506 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.419507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.419518 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.419550 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.420010 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.420495 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.421302 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.421510 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.421587 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.451887 (+ 30300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.451890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.451890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.451901 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.453784 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.454196 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.454198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.454198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.454252 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.454443 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.454443 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.454444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.454573 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.454615 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.454807 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.454914 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:44:24.455345 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.418869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:40663 (local address 127.0.0.1:60780)
0620 18:44:24.419298 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.419329 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.419411 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.419617 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.419617 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.419873 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.419918 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.419920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.420578 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.420579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.421212 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.421214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.421663 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.421664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.421723 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.422238 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.422243 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.451745 (+ 29502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.453881 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.453883 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.453883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.454114 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.454322 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.454323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.454323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.454378 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.454661 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.454662 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.454704 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.455106 (+ 402us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.455215 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":35,"thread_start_us":90,"threads_started":1}
I20260620 18:44:24.466282 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.419171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40136 (local address 127.15.77.126:38851)
0620 18:44:24.419220 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.419222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.419383 (+ 161us) server_negotiation.cc:407] Connection header received
0620 18:44:24.419506 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.419507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.419518 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.419550 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.451922 (+ 32372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.453356 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.454507 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.454799 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.454843 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.463286 (+ 8443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.463288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.463289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.463298 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.464958 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.465399 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.465400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.465400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.465465 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.465673 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.465673 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.465674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.465820 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.465887 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.466082 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.466165 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:44:24.466285 22516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.419189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40136)
0620 18:44:24.419329 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.419338 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.419411 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.419617 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.419617 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.451605 (+ 31988us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.451693 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.451697 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.453456 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.453457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.454396 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.454399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.454946 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.454947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.455070 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.462599 (+ 7529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.462605 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.463149 (+ 544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.465043 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.465045 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.465045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.465318 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.465533 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.465534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.465534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.465598 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.465885 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.465886 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.465938 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.465999 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.466149 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981063, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.126@KRBTEST.COM for oryx/127.15.77.124@KRBTEST.COM
I20260620 18:44:24.476650 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.469046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39841 (local address 127.0.0.1:52558)
0620 18:44:24.469097 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.469106 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.469140 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.469285 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.469286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.469449 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.469501 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.469502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.470397 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.470398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.471237 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.471239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.471722 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.471723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.471793 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.472207 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.472213 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.473748 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.475541 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.475542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.475543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.475759 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.475973 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.475974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.475974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.476046 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.476317 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.476318 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.476364 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.476437 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.476541 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 18:44:24.476652 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.469106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.15.77.124:39841)
0620 18:44:24.469144 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.469146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.469178 (+ 32us) server_negotiation.cc:407] Connection header received
0620 18:44:24.469210 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.469210 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.469220 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.469271 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.469576 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.470310 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.471344 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.471644 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.471716 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.473839 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.473841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.473841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.473849 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.475459 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.475836 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.475838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.475838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.475898 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.476112 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.476113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.476113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.476244 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.476312 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.476484 (+ 172us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.476559 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:44:24.477545 21773 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.77.124:39841: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519"
member_type: NON_VOTER
last_known_addr {
host: "127.15.77.124"
port: 39841
}
attrs {
promote: true
}
}
dest_uuid: "3afa68609e9b4416a496e310aba3a205"
cas_config_opid_index: 5
I20260620 18:44:24.477788 21773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:24.478114 22334 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5d433ee43d8b4dd58922cabbe3098519 (127.15.77.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.479051 22334 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.77.124:39841
I20260620 18:44:24.479116 22334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.479180 22334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:24.479701 22426 master.cc:537] Master@127.15.77.124:39841 shutting down...
I20260620 18:44:24.479789 22468 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 1 FOLLOWER]: Refusing update from remote peer 3afa68609e9b4416a496e310aba3a205: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 18:44:24.480261 22333 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 18:44:24.481276 21758 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 -> Peer 5d433ee43d8b4dd58922cabbe3098519 (127.15.77.124:39841): Couldn't send request to peer 5d433ee43d8b4dd58922cabbe3098519. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 18:44:24.482961 22426 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:24.483040 22426 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:24.483078 22426 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: stopping tablet replica
I20260620 18:44:24.485703 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.479427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:40663 (local address 127.0.0.1:60794)
0620 18:44:24.482102 (+ 2675us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.482115 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.482165 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.482395 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.482395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.482397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:24.482437 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.482439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.483388 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.483389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.484532 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.484536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.485439 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.485440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.485505 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.485532 (+ 27us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.485581 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.485621 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2656}
I20260620 18:44:24.486016 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.479517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60794 (local address 127.15.77.125:40663)
0620 18:44:24.479563 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.479565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.482225 (+ 2660us) server_negotiation.cc:407] Connection header received
0620 18:44:24.482256 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.482257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.482281 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.482334 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:24.482505 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.483252 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.484626 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.485201 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.485338 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.485406 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.485911 (+ 505us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.485955 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:44:24.486088 22426 master.cc:552] Master@127.15.77.124:39841 shutdown complete.
I20260620 18:44:24.486126 22426 master_runner.cc:305] Clearing existing system tablet
I20260620 18:44:24.486254 22426 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:24.486301 22426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:24.486444 22382 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 LEARNER]: Refusing update from remote peer 3afa68609e9b4416a496e310aba3a205: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 18:44:24.486629 22333 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 18:44:24.486991 22426 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:24.487162 22382 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5d433ee43d8b4dd58922cabbe3098519 (127.15.77.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.488372 22523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.488471 22523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: This master's current role is: LEARNER
I20260620 18:44:24.488603 22411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3afa68609e9b4416a496e310aba3a205. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: NON_VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: true } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.488692 22411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: This master's current role is: LEARNER
I20260620 18:44:24.488976 22522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.489157 22426 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:24.489228 22426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260620 18:44:24.489255 22426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal
uuid: "5d433ee43d8b4dd58922cabbe3098519"
format_stamp: "Formatted at 2026-06-20 18:44:24 on dist-test-slave-tvl9"
server_key: "12c25dc58f946e7885fa0029e29e0bdb"
server_key_iv: "49f002f9dc58fcb01e28ab385213e197"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.490026 22426 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:24.491508 22426 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 18:44:24.491578 22426 log.cc:1196] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260620 18:44:24.491779 22426 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: Deleting consensus metadata
I20260620 18:44:24.491897 22426 master_runner.cc:315] Copying system tablet from 127.15.77.126:38851
I20260620 18:44:24.491979 22333 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205: attempting to promote NON_VOTER 56cb9590fadb4cf68fc41eced963ab04 to VOTER
I20260620 18:44:24.492195 22333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:24.492527 22426 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: tablet copy: Beginning tablet copy session from remote peer at address 127.15.77.126:38851
I20260620 18:44:24.492846 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.492731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39841 (local address 127.0.0.1:52566)
0620 18:44:24.492775 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.492816 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 18:44:24.493204 22382 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 LEARNER]: Refusing update from remote peer 3afa68609e9b4416a496e310aba3a205: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260620 18:44:24.493572 22334 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260620 18:44:24.494196 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.494114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39841 (local address 127.0.0.1:52578)
0620 18:44:24.494147 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.494170 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260620 18:44:24.494315 21758 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 -> Peer 5d433ee43d8b4dd58922cabbe3098519 (127.15.77.124:39841): Couldn't send request to peer 5d433ee43d8b4dd58922cabbe3098519. Status: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 18:44:24.494714 22334 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 56cb9590fadb4cf68fc41eced963ab04 (127.15.77.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.495568 22382 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 56cb9590fadb4cf68fc41eced963ab04 (127.15.77.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.496488 22411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.496564 22411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:44:24.497294 22334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.497375 22334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:24.497435 22334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.497471 22334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:24.500540 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.492913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40144)
0620 18:44:24.493072 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.493081 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.493156 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.493359 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.493359 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.493564 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.493615 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.493617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.494589 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.494590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.495387 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.495390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.495895 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.495896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.495950 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.496522 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.496528 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.497067 (+ 539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.499225 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.499227 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.499227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.499569 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.499796 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.499797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.499797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.499869 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.500179 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.500180 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.500243 (+ 63us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.500319 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.500435 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":66,"threads_started":1}
I20260620 18:44:24.500543 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.492830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.15.77.126:38851)
0620 18:44:24.492871 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.492873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.493131 (+ 258us) server_negotiation.cc:407] Connection header received
0620 18:44:24.493253 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.493254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.493264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.493295 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.493703 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.494471 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.495485 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.495771 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.495813 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.497172 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.497173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.497174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.497181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.499134 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.499649 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.499650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.499651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.499718 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.499944 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.499945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.499945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.500103 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.500180 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.500364 (+ 184us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.500458 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:44:24.501259 21794 tablet_copy_service.cc:140] P 3afa68609e9b4416a496e310aba3a205: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5d433ee43d8b4dd58922cabbe3098519 ({username='oryx', principal='oryx/127.15.77.124@KRBTEST.COM'} at 127.0.0.1:40144)
I20260620 18:44:24.501317 21794 tablet_copy_service.cc:161] P 3afa68609e9b4416a496e310aba3a205: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5d433ee43d8b4dd58922cabbe3098519 at {username='oryx', principal='oryx/127.15.77.124@KRBTEST.COM'} at 127.0.0.1:40144: session id = 5d433ee43d8b4dd58922cabbe3098519-00000000000000000000000000000000
I20260620 18:44:24.501928 21794 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 18:44:24.502444 22426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 18:44:24.504127 22426 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: tablet copy: Starting download of 0 data blocks...
I20260620 18:44:24.504299 22426 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: tablet copy: Starting download of 1 WAL segments...
I20260620 18:44:24.505136 22426 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 18:44:24.506062 21794 tablet_copy_service.cc:342] P 3afa68609e9b4416a496e310aba3a205: Request end of tablet copy session 5d433ee43d8b4dd58922cabbe3098519-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.77.124@KRBTEST.COM'} at 127.0.0.1:40144
I20260620 18:44:24.506109 21794 tablet_copy_service.cc:433] P 3afa68609e9b4416a496e310aba3a205: ending tablet copy session 5d433ee43d8b4dd58922cabbe3098519-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5d433ee43d8b4dd58922cabbe3098519
I20260620 18:44:24.508543 22426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:24.509778 22532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:24.509800 22533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.509982 22426 server_base.cc:1034] running on GCE node
W20260620 18:44:24.510192 22535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.510388 22426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:24.510533 22426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:24.511694 22426 hybrid_clock.cc:648] HybridClock initialized: now 1781981064511681 us; error 29 us; skew 500 ppm
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:24.513109 22426 init.cc:377] Logged in from keytab as oryx/127.15.77.124@KRBTEST.COM (short username oryx)
I20260620 18:44:24.513411 22426 webserver.cc:460] Webserver started at http://127.15.77.124:39125/ using document root <none> and password file <none>
I20260620 18:44:24.513531 22426 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:24.513566 22426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:24.514168 22426 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:44:24.514695 22542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.514842 22426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.514889 22426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal
uuid: "5d433ee43d8b4dd58922cabbe3098519"
format_stamp: "Formatted at 2026-06-20 18:44:24 on dist-test-slave-tvl9"
server_key: "12c25dc58f946e7885fa0029e29e0bdb"
server_key_iv: "49f002f9dc58fcb01e28ab385213e197"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.514950 22426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:24.529917 22426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:24.531803 22426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:24.534868 22426 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.124:39841
I20260620 18:44:24.534891 22594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.124:39841 every 8 connection(s)
I20260620 18:44:24.535203 22426 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
I20260620 18:44:24.535955 22595 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:44:24.536648 22595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: Bootstrap starting.
I20260620 18:44:24.538996 22595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:44:24.539220 22595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: Bootstrap complete.
I20260620 18:44:24.539458 22595 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:24.539549 22595 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5d433ee43d8b4dd58922cabbe3098519, State: Initialized, Role: LEARNER
I20260620 18:44:24.539597 22595 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:24.539776 22595 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:24.539755 22598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:24.539960 22598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [sys.catalog]: This master's current role is: LEARNER
I20260620 18:44:24.541307 22609 catalog_manager.cc:1259] Loaded cluster ID: 07d816b82584432a9bbebaff644d5710
I20260620 18:44:24.541389 22609 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: loading cluster ID for follower catalog manager: success
I20260620 18:44:24.542683 22609 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: acquiring CA information for follower catalog manager: success
I20260620 18:44:24.542979 22609 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 18:44:24.572081 22519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:24.572383 22519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:24.572420 22519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:24.574258 22519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:24.574326 22519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:24.574368 22519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:44:24.576857 22519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:43161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab
--principal=oryx/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:41145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=37331
--tserver_master_addrs=127.15.77.126:38851,127.15.77.125:40663,127.15.77.124:39841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:24.577152 22519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:24.577421 22519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:24.583112 22615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.583128 22519 server_base.cc:1034] running on GCE node
W20260620 18:44:24.583344 22614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:24.583436 22617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.583674 22519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:24.583946 22519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:24.585114 22519 hybrid_clock.cc:648] HybridClock initialized: now 1781981064585103 us; error 34 us; skew 500 ppm
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:24.587695 22519 init.cc:377] Logged in from keytab as oryx/127.15.77.65@KRBTEST.COM (short username oryx)
I20260620 18:44:24.588094 22519 webserver.cc:460] Webserver started at http://127.15.77.65:37331/ using document root <none> and password file <none>
I20260620 18:44:24.588279 22519 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:24.588331 22519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:24.589722 22519 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:24.590261 22624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.590444 22519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.590536 22519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "c37364286d744b818ba2ce19ac421134"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "a08b8424bca1d59b36027d1f35f11d0d"
server_key_iv: "9506c25d40d80feb56d2d638c7a1c694"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.590878 22519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:24.604565 22519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:24.606541 22519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:24.606712 22519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:24.606964 22519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:24.607308 22519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:24.607400 22519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.607441 22519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:24.607467 22519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.613229 22519 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:41145
I20260620 18:44:24.613307 22737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:41145 every 8 connection(s)
I20260620 18:44:24.613654 22519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:44:24.619426 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 22519
I20260620 18:44:24.619614 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 21970
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.125@KRBTEST.COM
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.124@KRBTEST.COM
I20260620 18:44:24.626539 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:42623
--local_ip_for_outbound_sockets=127.15.77.66
--tserver_master_addrs=127.15.77.126:38851
--webserver_port=35623
--webserver_interface=127.15.77.66
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab
--principal=oryx/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:43161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.77.126:38851,127.15.77.125:40663
--tserver_master_addrs=127.15.77.126:38851,127.15.77.125:40663,127.15.77.124:39841 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:44:24.626679 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.614482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:48641 (local address 127.15.77.126:38851)
0620 18:44:24.614644 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.614646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.614879 (+ 233us) server_negotiation.cc:407] Connection header received
0620 18:44:24.616047 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.616048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.616060 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.616106 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.616984 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.617747 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.618653 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.618927 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.618993 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.623435 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.623437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.623437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.623445 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.625123 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.625556 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.625557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.625558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.625614 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.625914 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.625915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.625915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.626043 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.626089 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.626487 (+ 398us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.626568 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260620 18:44:24.627087 22742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.614396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:48641)
0620 18:44:24.614714 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.614761 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.615942 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.616208 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.616214 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.616634 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.616868 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.616874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.617839 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.617840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.618555 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.618557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.618999 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.619000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.619153 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.620173 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.620184 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.623337 (+ 3153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.625215 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.625216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.625216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.625468 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.625690 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.625691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.625691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.625833 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.626137 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.626137 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.626308 (+ 171us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.626809 (+ 501us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.626900 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":12,"thread_start_us":91,"threads_started":1}
I20260620 18:44:24.627262 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.614396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39841 (local address 127.15.77.65:33209)
0620 18:44:24.614879 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.614890 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.615965 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.616213 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.616214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.616663 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.616868 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.616874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.618106 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.618107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.618864 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.618866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.619365 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.619366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.620384 (+ 1018us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.620829 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.620835 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.622937 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.625070 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.625078 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.625080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.625468 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.625694 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.625695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.625695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.625832 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.626130 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.626137 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.626308 (+ 171us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.626437 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.626838 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":39,"thread_start_us":94,"threads_started":1}
I20260620 18:44:24.627526 22745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.614728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:33209 (local address 127.15.77.124:39841)
0620 18:44:24.614858 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.614860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.614952 (+ 92us) server_negotiation.cc:407] Connection header received
0620 18:44:24.616109 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.616110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.616122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.616199 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.616979 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.618015 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.618941 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.619235 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.621446 (+ 2211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.623053 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.623054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.623055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.623064 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.624973 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.625550 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.625551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.625551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.625614 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.625906 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.625908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.625908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.626044 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.626103 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.626971 (+ 868us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.627052 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":56,"threads_started":1}
I20260620 18:44:24.628182 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.614541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:57191 (local address 127.15.77.125:40663)
0620 18:44:24.614685 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.614688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.615031 (+ 343us) server_negotiation.cc:407] Connection header received
0620 18:44:24.616068 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.616069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.616082 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.616113 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.616985 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.617476 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.618655 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.618987 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.619027 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.623893 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.623895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.623895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.623904 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.626116 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.626554 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.626556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.626556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.626614 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.627124 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.627125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.627126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.627299 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.627370 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.627968 (+ 598us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.628062 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260620 18:44:24.628191 22738 heartbeater.cc:347] Connected to a master server at 127.15.77.124:39841
I20260620 18:44:24.628299 22738 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:24.628355 22739 heartbeater.cc:347] Connected to a master server at 127.15.77.126:38851
I20260620 18:44:24.628401 22739 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:24.628494 22738 heartbeater.cc:511] Master 127.15.77.124:39841 requested a full tablet report, sending...
I20260620 18:44:24.628499 22739 heartbeater.cc:511] Master 127.15.77.126:38851 requested a full tablet report, sending...
I20260620 18:44:24.628510 22744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.614396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:40663 (local address 127.15.77.65:57191)
0620 18:44:24.614989 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.614995 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.615960 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.616214 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.616215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.616646 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.616868 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.616873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.617585 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.617586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.618558 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.618560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.619105 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.619106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.619160 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.620173 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.620184 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.623592 (+ 3408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.626227 (+ 2635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.626228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.626228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.626467 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.626909 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.626910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.626910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.626991 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.627701 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.627702 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.627751 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.628273 (+ 522us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.628386 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":27,"thread_start_us":99,"threads_started":1}
I20260620 18:44:24.628957 21773 ts_manager.cc:194] Re-registered known tserver with Master: c37364286d744b818ba2ce19ac421134 (127.15.77.65:41145)
I20260620 18:44:24.629009 22740 heartbeater.cc:347] Connected to a master server at 127.15.77.125:40663
I20260620 18:44:24.629060 22740 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:24.629127 22740 heartbeater.cc:511] Master 127.15.77.125:40663 requested a full tablet report, sending...
I20260620 18:44:24.629113 22559 ts_manager.cc:194] Registered new tserver with Master: c37364286d744b818ba2ce19ac421134 (127.15.77.65:41145)
I20260620 18:44:24.629444 21773 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:48641
I20260620 18:44:24.629469 22372 ts_manager.cc:194] Registered new tserver with Master: c37364286d744b818ba2ce19ac421134 (127.15.77.65:41145)
W20260620 18:44:24.711295 22746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:24.711589 22746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:24.711614 22746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:24.712757 22746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:24.712793 22746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:24.712819 22746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:44:24.714334 22746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:43161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab
--principal=oryx/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:42623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=35623
--tserver_master_addrs=127.15.77.126:38851,127.15.77.125:40663,127.15.77.124:39841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:24.714593 22746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:24.714787 22746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:24.719851 22751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.720031 22746 server_base.cc:1034] running on GCE node
W20260620 18:44:24.719880 22754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:24.719898 22752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.720387 22746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:24.720643 22746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:24.721787 22746 hybrid_clock.cc:648] HybridClock initialized: now 1781981064721766 us; error 37 us; skew 500 ppm
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:24.724248 22746 init.cc:377] Logged in from keytab as oryx/127.15.77.66@KRBTEST.COM (short username oryx)
I20260620 18:44:24.724700 22746 webserver.cc:460] Webserver started at http://127.15.77.66:35623/ using document root <none> and password file <none>
I20260620 18:44:24.724931 22746 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:24.725010 22746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:24.726356 22746 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:24.727015 22761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.727205 22746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:24.727306 22746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "1f7bc557402b462d94288962ae3a2be5"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "80790ba1381d5dc80093095e58be1fc6"
server_key_iv: "dc14fb9e69dfe4bda8eda036ac9653e7"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.727722 22746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:24.742841 22746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:24.744949 22746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:24.745134 22746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:24.745390 22746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:24.745723 22746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:24.745779 22746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.745820 22746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:24.745848 22746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.752060 22746 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:42623
I20260620 18:44:24.752126 22874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:42623 every 8 connection(s)
I20260620 18:44:24.752465 22746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:44:24.753067 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 22746
I20260620 18:44:24.753254 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 22104
I20260620 18:44:24.760485 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:45993
--local_ip_for_outbound_sockets=127.15.77.67
--tserver_master_addrs=127.15.77.126:38851
--webserver_port=45521
--webserver_interface=127.15.77.67
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab
--principal=oryx/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:43161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.77.126:38851,127.15.77.125:40663
--tserver_master_addrs=127.15.77.126:38851,127.15.77.125:40663,127.15.77.124:39841 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.124@KRBTEST.COM
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.125@KRBTEST.COM
I20260620 18:44:24.765998 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.753737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:59049 (local address 127.15.77.126:38851)
0620 18:44:24.753798 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.753801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.754036 (+ 235us) server_negotiation.cc:407] Connection header received
0620 18:44:24.754587 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.754588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.754601 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.754675 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.756131 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.756621 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.758218 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.758397 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.758449 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.761911 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.761913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.761913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.761923 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.763874 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.764523 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.764525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.764525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.764611 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.765004 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.765005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.765006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.765178 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.765227 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.765758 (+ 531us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.765861 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 18:44:24.766580 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.753249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:59049)
0620 18:44:24.753872 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.753912 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.754478 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.754814 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.754819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.755705 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.756004 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.756012 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.756730 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.756732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.758131 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.758133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.758492 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.758493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.758661 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.759363 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.759374 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.761778 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.763971 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.763980 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.763983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.764431 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.764694 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.764695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.764695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.764852 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.765339 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.765347 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.765544 (+ 197us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.765720 (+ 176us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.765923 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":88,"threads_started":1}
I20260620 18:44:24.767189 22876 heartbeater.cc:347] Connected to a master server at 127.15.77.126:38851
I20260620 18:44:24.767292 22876 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:24.767534 22876 heartbeater.cc:511] Master 127.15.77.126:38851 requested a full tablet report, sending...
I20260620 18:44:24.767956 21773 ts_manager.cc:194] Re-registered known tserver with Master: 1f7bc557402b462d94288962ae3a2be5 (127.15.77.66:42623)
I20260620 18:44:24.768493 21773 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:59049
I20260620 18:44:24.768855 22745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.759436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:48437 (local address 127.15.77.124:39841)
0620 18:44:24.759517 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.759519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.759681 (+ 162us) server_negotiation.cc:407] Connection header received
0620 18:44:24.759779 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.759780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.759796 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.759827 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.760301 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.761047 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.762046 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.762346 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.762390 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.764735 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.764737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.764737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.764745 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.767387 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.767907 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.767909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.767909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.767973 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.768236 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.768237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.768237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.768364 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.768403 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.768654 (+ 251us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.768762 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 18:44:24.768963 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.753538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:41547 (local address 127.15.77.125:40663)
0620 18:44:24.753582 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.753584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.759481 (+ 5897us) server_negotiation.cc:407] Connection header received
0620 18:44:24.759593 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.759595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.759609 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.759643 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.760201 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.760955 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.762046 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.763161 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.763230 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.765293 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.765295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.765295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.765304 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.767495 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.767995 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.767997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.767997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.768062 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.768285 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.768286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.768287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.768435 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.768494 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.768738 (+ 244us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.768830 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:44:24.769402 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.753249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:40663 (local address 127.15.77.66:41547)
0620 18:44:24.759395 (+ 6146us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.759407 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.759513 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.759714 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.759715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.760007 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.760084 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.760085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.761047 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.761048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.761944 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.761946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.763287 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.763288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.763377 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.763790 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.763795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.765184 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.767608 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.767609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.767609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.767897 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.768145 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.768146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.768146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.768208 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.768548 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.768549 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.768609 (+ 60us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.769085 (+ 476us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.769249 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6025,"mutex_wait_us":37,"spinlock_wait_cycles":8576,"thread_start_us":98,"threads_started":1}
I20260620 18:44:24.769588 22881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.759285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39841 (local address 127.15.77.66:48437)
0620 18:44:24.759616 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.759625 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.759706 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.759888 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.759889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.760140 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.760215 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.760217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.761159 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.761160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.761948 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.761949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.762473 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.762474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.762544 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.762993 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.762998 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.764623 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.767482 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.767483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.767483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.767820 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.768090 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.768091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.768091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.768163 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.768444 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.768445 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.768485 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.769299 (+ 814us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.769424 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":4,"thread_start_us":260,"threads_started":1}
I20260620 18:44:24.769961 22875 heartbeater.cc:347] Connected to a master server at 127.15.77.124:39841
I20260620 18:44:24.770020 22875 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:24.770036 22877 heartbeater.cc:347] Connected to a master server at 127.15.77.125:40663
I20260620 18:44:24.770074 22877 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:24.770107 22875 heartbeater.cc:511] Master 127.15.77.124:39841 requested a full tablet report, sending...
I20260620 18:44:24.770112 22877 heartbeater.cc:511] Master 127.15.77.125:40663 requested a full tablet report, sending...
I20260620 18:44:24.770486 22372 ts_manager.cc:194] Registered new tserver with Master: 1f7bc557402b462d94288962ae3a2be5 (127.15.77.66:42623)
I20260620 18:44:24.770536 22559 ts_manager.cc:194] Registered new tserver with Master: 1f7bc557402b462d94288962ae3a2be5 (127.15.77.66:42623)
W20260620 18:44:24.848596 22882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:24.848845 22882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:24.848886 22882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:24.850123 22882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:24.850191 22882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:24.850237 22882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:44:24.851752 22882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:43161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab
--principal=oryx/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:45993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=45521
--tserver_master_addrs=127.15.77.126:38851,127.15.77.125:40663,127.15.77.124:39841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:24.852033 22882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:24.852284 22882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:24.857565 22890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:24.857565 22888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:24.857577 22887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:24.857772 22882 server_base.cc:1034] running on GCE node
I20260620 18:44:24.858006 22882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:24.858268 22882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:24.859441 22882 hybrid_clock.cc:648] HybridClock initialized: now 1781981064859415 us; error 56 us; skew 500 ppm
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:24.862002 22882 init.cc:377] Logged in from keytab as oryx/127.15.77.67@KRBTEST.COM (short username oryx)
I20260620 18:44:24.862416 22882 webserver.cc:460] Webserver started at http://127.15.77.67:45521/ using document root <none> and password file <none>
I20260620 18:44:24.862628 22882 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:24.862694 22882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:24.864147 22882 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:24.864737 22897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.864923 22882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:24.865005 22882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "5e4de4da6a02462b80bdfbde421dc0a6"
format_stamp: "Formatted at 2026-06-20 18:44:23 on dist-test-slave-tvl9"
server_key: "339d51a5879ec070ee2a2d93c514733a"
server_key_iv: "f7f87debde1e36b2d8417a25b1fda94f"
server_key_version: "encryptionkey@0"
I20260620 18:44:24.865437 22882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:24.889657 22882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:24.892163 22882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:24.892328 22882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:24.892602 22882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:44:24.892944 22882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:24.893002 22882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.893044 22882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:24.893083 22882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:24.899565 22882 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:45993
I20260620 18:44:24.899610 23010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:45993 every 8 connection(s)
I20260620 18:44:24.900000 22882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:44:24.906843 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 22882
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.125@KRBTEST.COM
Jun 20 18:44:24 dist-test-slave-tvl9 krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981064, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.124@KRBTEST.COM
I20260620 18:44:24.912962 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.901242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:40663 (local address 127.15.77.67:49165)
0620 18:44:24.901381 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.901392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.901736 (+ 344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.902009 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.902010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.902555 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.902736 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.902742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.903710 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.903710 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.904917 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.904920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.905773 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.905773 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.905834 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.906386 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.906391 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.908716 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.911206 (+ 2490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.911215 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.911218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.911563 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.911767 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.911768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.911768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.911913 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.912226 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.912232 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.912401 (+ 169us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.912514 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.912697 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":30,"thread_start_us":78,"threads_started":1}
I20260620 18:44:24.912963 22745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.901135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:43275 (local address 127.15.77.124:39841)
0620 18:44:24.901182 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.901184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.901194 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:24.901855 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.901856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.901869 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.901902 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.902842 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.903634 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.905744 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.906037 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.906107 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.909795 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.909796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.909797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.909803 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.911430 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.911870 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.911871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.911871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.911932 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.912167 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.912168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.912168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.912388 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.912509 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.912754 (+ 245us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.912839 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:44:24.913022 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.900831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:54117 (local address 127.15.77.126:38851)
0620 18:44:24.900909 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.900911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.901150 (+ 239us) server_negotiation.cc:407] Connection header received
0620 18:44:24.901848 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.901849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.901862 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.901915 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.902839 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.903403 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.905016 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.905327 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.905398 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.909126 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.909127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.909128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.909135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.911477 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.911935 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.911936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.911936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.911995 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.912458 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.912459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.912460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.912603 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.912647 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.912848 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.912921 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 18:44:24.913317 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.901314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:49165 (local address 127.15.77.125:40663)
0620 18:44:24.901366 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.901368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.901440 (+ 72us) server_negotiation.cc:407] Connection header received
0620 18:44:24.901855 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.901856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.901869 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.901902 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.902842 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.903630 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.905013 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.905326 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.905398 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.908809 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.908811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.908812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.908819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.911112 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.911640 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.911642 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.911642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.911698 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.911985 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.911985 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.911986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.912138 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.912200 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.913104 (+ 904us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.913193 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:44:24.914294 23016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.900800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:54117)
0620 18:44:24.901019 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.901062 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.901736 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.902269 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.902269 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.902509 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.902734 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.902738 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.903509 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.903510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.904917 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.904920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.905416 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.905417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.905556 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.906225 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.906236 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.909017 (+ 2781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.911565 (+ 2548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.911565 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.911566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.911851 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.912070 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.912071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.912071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.912124 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.912698 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.912699 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.912735 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.913481 (+ 746us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.913601 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":66,"thread_start_us":147,"threads_started":1}
I20260620 18:44:24.914784 23012 heartbeater.cc:347] Connected to a master server at 127.15.77.126:38851
I20260620 18:44:24.914948 23012 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:24.915143 23012 heartbeater.cc:511] Master 127.15.77.126:38851 requested a full tablet report, sending...
I20260620 18:44:24.915608 21773 ts_manager.cc:194] Re-registered known tserver with Master: 5e4de4da6a02462b80bdfbde421dc0a6 (127.15.77.67:45993)
I20260620 18:44:24.913931 23013 heartbeater.cc:347] Connected to a master server at 127.15.77.125:40663
I20260620 18:44:24.915714 23013 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:24.915786 23013 heartbeater.cc:511] Master 127.15.77.125:40663 requested a full tablet report, sending...
I20260620 18:44:24.916090 21773 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:54117
I20260620 18:44:24.916368 22372 ts_manager.cc:194] Registered new tserver with Master: 5e4de4da6a02462b80bdfbde421dc0a6 (127.15.77.67:45993)
I20260620 18:44:24.916909 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.907970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40158 (local address 127.15.77.126:38851)
0620 18:44:24.908539 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.908541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.908548 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:44:24.908570 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.908571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.908582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.908650 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.909566 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.910356 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.911487 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.911839 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.911878 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.913441 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.913442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.913443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.913449 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.915492 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.915932 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.915934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.915934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.916004 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.916281 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.916283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.916283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.916413 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.916512 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.916732 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.916816 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":120,"threads_started":1}
I20260620 18:44:24.917009 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.900799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39841 (local address 127.15.77.67:43275)
0620 18:44:24.901017 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.901062 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.901738 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.901992 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.901997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.902540 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:24.902736 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.902742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.905025 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.905026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.905653 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.905655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.906115 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.906116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.906215 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.906559 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:24.906564 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:24.909704 (+ 3140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:24.911512 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.911513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.911514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.911776 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.912006 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:24.912006 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:24.912007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:24.912070 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:24.912575 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:24.912576 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:24.912623 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.913762 (+ 1139us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.913867 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":38,"thread_start_us":120,"threads_started":1}
I20260620 18:44:24.917335 23011 heartbeater.cc:347] Connected to a master server at 127.15.77.124:39841
I20260620 18:44:24.917375 23011 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:24.917431 23011 heartbeater.cc:511] Master 127.15.77.124:39841 requested a full tablet report, sending...
I20260620 18:44:24.917735 22559 ts_manager.cc:194] Registered new tserver with Master: 5e4de4da6a02462b80bdfbde421dc0a6 (127.15.77.67:45993)
I20260620 18:44:24.919088 21773 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:44:24.921423 21773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:24.923197 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.908190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52594 (local address 127.15.77.124:39841)
0620 18:44:24.909091 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.909093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.909101 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:44:24.909123 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.909124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.909135 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.910212 (+ 1077us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.910253 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.914140 (+ 3887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.915462 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.915725 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.915789 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.918669 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.918670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.918671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.918678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.920352 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.922292 (+ 1940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.922294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.922294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.922358 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.922587 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.922588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.922588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.922736 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.922853 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.922996 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.923087 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":61,"threads_started":1}
I20260620 18:44:24.924422 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.908095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.15.77.125:40663)
0620 18:44:24.908721 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.908723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.914343 (+ 5620us) server_negotiation.cc:407] Connection header received
0620 18:44:24.914448 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.914449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.914464 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.914506 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:24.916797 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.917483 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.919013 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.919277 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.919345 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.920850 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:24.920851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:24.920852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:24.920858 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:24.922617 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.922974 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.922975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.922976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.923037 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:24.923239 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:24.923239 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:24.923240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:24.923419 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:24.923515 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.924240 (+ 725us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.924324 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":40,"threads_started":1}
I20260620 18:44:24.929498 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.925561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:41145 (local address 127.0.0.1:47878)
0620 18:44:24.925638 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.925648 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.925686 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.926138 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.926138 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.926140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:24.926184 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.926185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.927408 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.927409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.928497 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.928498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.929259 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.929260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.929325 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.929329 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.929385 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.929429 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260620 18:44:24.929513 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.925677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47878 (local address 127.15.77.65:41145)
0620 18:44:24.925971 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.925977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.925989 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:44:24.926032 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.926034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.926051 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.926142 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:24.926238 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.927272 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.928593 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.929162 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.929250 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.929328 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.929399 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.929434 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":131,"threads_started":1}
I20260620 18:44:24.930195 22672 tablet_service.cc:1467] Processing CreateTablet for tablet 9f719b7d2f624ba4b92a69e2f8e01bf8 (DEFAULT_TABLE table=test-table [id=9f09dd19f9724d28a8db231afb41d5e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:24.930603 22672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f719b7d2f624ba4b92a69e2f8e01bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:24.931200 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.925848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:45993 (local address 127.0.0.1:52216)
0620 18:44:24.926671 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.926679 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.926723 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.926916 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.926917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.926919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:24.926963 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.926964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.928480 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.928481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.929782 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.929785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.930948 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.930949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.931012 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.931018 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.931075 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.931122 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":60,"threads_started":1}
I20260620 18:44:24.931281 23036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.926296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.15.77.67:45993)
0620 18:44:24.926489 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.926493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.926763 (+ 270us) server_negotiation.cc:407] Connection header received
0620 18:44:24.926807 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.926809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.926822 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.926898 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:24.927043 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.928388 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.929874 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.930861 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.930948 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.931003 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.931128 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.931167 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260620 18:44:24.931336 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.925682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:42623 (local address 127.0.0.1:44436)
0620 18:44:24.926547 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:24.926557 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:24.926601 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:24.927611 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:24.927612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:24.927613 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:24.927656 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.927657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.928747 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.928748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.930040 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:24.930042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.931078 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.931080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:24.931145 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.931149 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:44:24.931202 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 18:44:24.931240 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"thread_start_us":83,"threads_started":1}
I20260620 18:44:24.931555 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:24.926132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44436 (local address 127.15.77.66:42623)
0620 18:44:24.927466 (+ 1334us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:24.927470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:24.927478 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:44:24.927508 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:24.927509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:24.927522 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:24.927612 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:24.927711 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.928661 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.930125 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:24.930893 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:24.930966 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:24.931039 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:24.931408 (+ 369us) server_negotiation.cc:299] Negotiation successful
0620 18:44:24.931484 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1290,"thread_start_us":79,"threads_started":1}
I20260620 18:44:24.933561 23037 tablet_bootstrap.cc:492] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: Bootstrap starting.
I20260620 18:44:24.934307 23037 tablet_bootstrap.cc:654] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:24.934331 22945 tablet_service.cc:1467] Processing CreateTablet for tablet 9f719b7d2f624ba4b92a69e2f8e01bf8 (DEFAULT_TABLE table=test-table [id=9f09dd19f9724d28a8db231afb41d5e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:24.934415 22809 tablet_service.cc:1467] Processing CreateTablet for tablet 9f719b7d2f624ba4b92a69e2f8e01bf8 (DEFAULT_TABLE table=test-table [id=9f09dd19f9724d28a8db231afb41d5e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:24.934649 23037 log.cc:824] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:24.934738 22945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f719b7d2f624ba4b92a69e2f8e01bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:24.934784 22809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f719b7d2f624ba4b92a69e2f8e01bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:24.937600 23039 tablet_bootstrap.cc:492] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: Bootstrap starting.
I20260620 18:44:24.938040 23037 tablet_bootstrap.cc:492] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: No bootstrap required, opened a new log
I20260620 18:44:24.938100 23037 ts_tablet_manager.cc:1397] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260620 18:44:24.938207 23039 tablet_bootstrap.cc:654] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:24.938491 23039 log.cc:824] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:24.938849 23040 tablet_bootstrap.cc:492] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: Bootstrap starting.
I20260620 18:44:24.939249 23039 tablet_bootstrap.cc:492] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: No bootstrap required, opened a new log
I20260620 18:44:24.939301 23039 ts_tablet_manager.cc:1397] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:44:24.940388 23037 raft_consensus.cc:348] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 } } peers { permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 } } peers { permanent_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45993 } }
I20260620 18:44:24.940588 23037 raft_consensus.cc:374] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:24.940789 23037 raft_consensus.cc:729] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37364286d744b818ba2ce19ac421134, State: Initialized, Role: FOLLOWER
I20260620 18:44:24.940930 23037 consensus_queue.cc:260] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 } } peers { permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 } } peers { permanent_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45993 } }
I20260620 18:44:24.941054 23039 raft_consensus.cc:348] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 } } peers { permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 } } peers { permanent_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45993 } }
I20260620 18:44:24.941202 23037 ts_tablet_manager.cc:1428] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:44:24.941217 23039 raft_consensus.cc:374] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:24.941251 22739 heartbeater.cc:503] Master 127.15.77.126:38851 was elected leader, sending a full tablet report...
I20260620 18:44:24.941255 23039 raft_consensus.cc:729] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4de4da6a02462b80bdfbde421dc0a6, State: Initialized, Role: FOLLOWER
I20260620 18:44:24.941363 23039 consensus_queue.cc:260] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 } } peers { permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 } } peers { permanent_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45993 } }
I20260620 18:44:24.941614 23039 ts_tablet_manager.cc:1428] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:44:24.941736 23012 heartbeater.cc:503] Master 127.15.77.126:38851 was elected leader, sending a full tablet report...
I20260620 18:44:24.942342 23040 tablet_bootstrap.cc:654] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:24.942670 23040 log.cc:824] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:24.943472 23040 tablet_bootstrap.cc:492] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: No bootstrap required, opened a new log
I20260620 18:44:24.943540 23040 ts_tablet_manager.cc:1397] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260620 18:44:24.944856 23040 raft_consensus.cc:348] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 } } peers { permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 } } peers { permanent_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45993 } }
I20260620 18:44:24.944998 23040 raft_consensus.cc:374] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:24.945043 23040 raft_consensus.cc:729] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f7bc557402b462d94288962ae3a2be5, State: Initialized, Role: FOLLOWER
I20260620 18:44:24.945156 23040 consensus_queue.cc:260] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 } } peers { permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 } } peers { permanent_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45993 } }
I20260620 18:44:24.945393 23040 ts_tablet_manager.cc:1428] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:44:24.945463 22876 heartbeater.cc:503] Master 127.15.77.126:38851 was elected leader, sending a full tablet report...
W20260620 18:44:25.005157 22878 tablet.cc:2369] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:44:25.048190 23043 raft_consensus.cc:482] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:25.048378 23043 raft_consensus.cc:504] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 } } peers { permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 } } peers { permanent_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45993 } }
I20260620 18:44:25.048831 23043 leader_election.cc:288] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c37364286d744b818ba2ce19ac421134 (127.15.77.65:41145), 1f7bc557402b462d94288962ae3a2be5 (127.15.77.66:42623)
I20260620 18:44:25.052085 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:25.049175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:32845 (local address 127.15.77.65:41145)
0620 18:44:25.049234 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:25.049236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:25.049247 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:44:25.049320 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:25.049321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:25.049332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:25.049392 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:25.049868 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.050335 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.051260 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.051712 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.051822 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:25.051832 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:25.051980 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 18:44:25.052020 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 18:44:25.052085 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:25.049049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:41145 (local address 127.15.77.67:32845)
0620 18:44:25.049157 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:25.049172 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:25.049246 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:25.049443 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:25.049444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:25.049446 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:25.049781 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:25.049783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.050419 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.050420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:25.051165 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:25.051167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.051821 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.051822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:25.051893 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:25.051896 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:44:25.051971 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 18:44:25.052009 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260620 18:44:25.052207 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:25.049250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:36191 (local address 127.15.77.66:42623)
0620 18:44:25.049292 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:25.049294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:25.049304 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:25.049495 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:25.049496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:25.049508 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:25.049591 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:25.049777 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.050313 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.051351 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.051806 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.051907 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:25.051915 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:25.052098 (+ 183us) server_negotiation.cc:299] Negotiation successful
0620 18:44:25.052138 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:44:25.052229 23016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:25.049180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:42623 (local address 127.15.77.67:36191)
0620 18:44:25.049229 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:25.049240 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:25.049390 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:25.049614 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:25.049616 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:25.049618 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:25.049692 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:25.049694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.050410 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.050412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:25.051232 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:25.051234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.051912 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.051913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:25.052011 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:25.052014 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:44:25.052090 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 18:44:25.052131 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":30}
I20260620 18:44:25.052591 22692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f719b7d2f624ba4b92a69e2f8e01bf8" candidate_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37364286d744b818ba2ce19ac421134" is_pre_election: true
I20260620 18:44:25.052683 22829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f719b7d2f624ba4b92a69e2f8e01bf8" candidate_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f7bc557402b462d94288962ae3a2be5" is_pre_election: true
I20260620 18:44:25.052771 22692 raft_consensus.cc:2454] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e4de4da6a02462b80bdfbde421dc0a6 in term 0.
I20260620 18:44:25.052872 22829 raft_consensus.cc:2454] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e4de4da6a02462b80bdfbde421dc0a6 in term 0.
I20260620 18:44:25.053035 22898 leader_election.cc:302] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e4de4da6a02462b80bdfbde421dc0a6, c37364286d744b818ba2ce19ac421134; no voters:
I20260620 18:44:25.053200 23043 raft_consensus.cc:2781] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:25.053280 23043 raft_consensus.cc:482] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:25.053316 23043 raft_consensus.cc:3037] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:25.054018 23043 raft_consensus.cc:504] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 } } peers { permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 } } peers { permanent_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45993 } }
I20260620 18:44:25.054169 23043 leader_election.cc:288] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers c37364286d744b818ba2ce19ac421134 (127.15.77.65:41145), 1f7bc557402b462d94288962ae3a2be5 (127.15.77.66:42623)
I20260620 18:44:25.054358 22692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f719b7d2f624ba4b92a69e2f8e01bf8" candidate_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37364286d744b818ba2ce19ac421134"
I20260620 18:44:25.054414 22829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f719b7d2f624ba4b92a69e2f8e01bf8" candidate_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f7bc557402b462d94288962ae3a2be5"
I20260620 18:44:25.054441 22692 raft_consensus.cc:3037] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:25.054477 22829 raft_consensus.cc:3037] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:25.055364 22692 raft_consensus.cc:2454] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4de4da6a02462b80bdfbde421dc0a6 in term 1.
I20260620 18:44:25.055363 22829 raft_consensus.cc:2454] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e4de4da6a02462b80bdfbde421dc0a6 in term 1.
I20260620 18:44:25.055536 22898 leader_election.cc:302] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f7bc557402b462d94288962ae3a2be5, 5e4de4da6a02462b80bdfbde421dc0a6; no voters:
I20260620 18:44:25.055640 23043 raft_consensus.cc:2781] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:25.055783 23043 raft_consensus.cc:686] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 5e4de4da6a02462b80bdfbde421dc0a6, State: Running, Role: LEADER
I20260620 18:44:25.055882 23043 consensus_queue.cc:237] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 } } peers { permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 } } peers { permanent_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45993 } }
I20260620 18:44:25.056834 21774 catalog_manager.cc:5495] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e4de4da6a02462b80bdfbde421dc0a6 (127.15.77.67). New cstate: current_term: 1 leader_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e4de4da6a02462b80bdfbde421dc0a6" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45993 } health_report { overall_health: HEALTHY } } }
I20260620 18:44:25.073441 23036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:25.071121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.15.77.67:45993)
0620 18:44:25.071178 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:25.071180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:25.071189 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:44:25.071232 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:25.071232 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:25.071250 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:25.071349 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:25.071485 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.071968 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.072694 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.072875 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.072925 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:25.073059 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:25.073188 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:25.073273 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:25.073324 (+ 51us) server_negotiation.cc:299] Negotiation successful
0620 18:44:25.073350 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 18:44:25.075439 22829 raft_consensus.cc:1261] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5 [term 1 FOLLOWER]: Refusing update from remote peer 5e4de4da6a02462b80bdfbde421dc0a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:25.075444 22692 raft_consensus.cc:1261] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134 [term 1 FOLLOWER]: Refusing update from remote peer 5e4de4da6a02462b80bdfbde421dc0a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:25.075800 23046 consensus_queue.cc:1035] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37364286d744b818ba2ce19ac421134" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:25.075971 23043 consensus_queue.cc:1035] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f7bc557402b462d94288962ae3a2be5" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:25.076614 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:25.073425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39841 (local address 127.0.0.1:52602)
0620 18:44:25.073486 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:25.073498 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:25.073540 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:25.073740 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:25.073741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:25.073743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:25.073791 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:25.073792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.074535 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.074536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:25.075518 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:25.075522 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.076363 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.076364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:25.076424 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:25.076428 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:44:25.076489 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 18:44:25.076539 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 18:44:25.076646 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:25.073525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.15.77.124:39841)
0620 18:44:25.073574 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:25.073576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:25.073584 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:44:25.073628 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:25.073629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:25.073668 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:25.073736 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:25.073873 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.074447 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.075635 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.076272 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.076363 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:25.076443 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:25.076505 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 18:44:25.076581 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:44:25.076929 22569 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.10] Dedup: 1.7->[1.8-1.10]
I20260620 18:44:25.077975 23051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3afa68609e9b4416a496e310aba3a205. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:25.078102 23051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [sys.catalog]: This master's current role is: LEARNER
I20260620 18:44:25.080145 23050 mvcc.cc:205] Tried to move back new op lower bound from 7298994442545938432 to 7298994442470060032. Current Snapshot: MvccSnapshot[applied={T|T < 7298994442545938432}]
I20260620 18:44:25.081808 23055 mvcc.cc:205] Tried to move back new op lower bound from 7298994442545938432 to 7298994442470060032. Current Snapshot: MvccSnapshot[applied={T|T < 7298994442545938432 or (T in {7298994442545938432})}]
I20260620 18:44:25.082448 21774 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:25.082538 21774 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:25.083112 22531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3afa68609e9b4416a496e310aba3a205" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } } }
I20260620 18:44:25.083213 22531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:25.084192 22528 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205: attempting to promote NON_VOTER 5d433ee43d8b4dd58922cabbe3098519 to VOTER
I20260620 18:44:25.084311 22528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: false } }
I20260620 18:44:25.084674 21774 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205: Sending DeleteTablet for 3 replicas of tablet 9f719b7d2f624ba4b92a69e2f8e01bf8
I20260620 18:44:25.084933 22381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Refusing update from remote peer 3afa68609e9b4416a496e310aba3a205: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20260620 18:44:25.085220 22672 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f719b7d2f624ba4b92a69e2f8e01bf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:25 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:47878
I20260620 18:44:25.085301 22809 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f719b7d2f624ba4b92a69e2f8e01bf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:25 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:44436
I20260620 18:44:25.085538 22945 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f719b7d2f624ba4b92a69e2f8e01bf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:25 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:52216
I20260620 18:44:25.085975 22569 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d433ee43d8b4dd58922cabbe3098519 [term 1 LEARNER]: Refusing update from remote peer 3afa68609e9b4416a496e310aba3a205: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20260620 18:44:25.086026 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 21742
I20260620 18:44:25.086082 22333 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3afa68609e9b4416a496e310aba3a205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260620 18:44:25.091437 23066 tablet_replica.cc:330] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: stopping tablet replica
I20260620 18:44:25.091476 23068 tablet_replica.cc:330] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: stopping tablet replica
I20260620 18:44:25.091595 23066 raft_consensus.cc:2229] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:25.091599 23068 raft_consensus.cc:2229] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:44:25.091732 23066 raft_consensus.cc:2258] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:25.091735 23068 raft_consensus.cc:2258] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:25.091990 23065 tablet_replica.cc:330] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: stopping tablet replica
I20260620 18:44:25.092108 23065 raft_consensus.cc:2229] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:25.092228 23065 raft_consensus.cc:2258] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:25.092293 23068 ts_tablet_manager.cc:1905] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 18:44:25.092500 22545 connection.cc:535] server connection from 127.0.0.1:52602 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 18:44:25.094295 23068 ts_tablet_manager.cc:1918] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:25.094381 23068 log.cc:1196] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/9f719b7d2f624ba4b92a69e2f8e01bf8
I20260620 18:44:25.094710 23068 ts_tablet_manager.cc:1939] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 5e4de4da6a02462b80bdfbde421dc0a6: Deleting consensus metadata
W20260620 18:44:25.095193 22901 connection.cc:441] server connection from 127.0.0.1:52216 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52216 (request call id 1) could send its response
I20260620 18:44:25.095216 23066 ts_tablet_manager.cc:1905] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:25.096824 23066 ts_tablet_manager.cc:1918] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:25.096915 23066 log.cc:1196] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/9f719b7d2f624ba4b92a69e2f8e01bf8
I20260620 18:44:25.097180 23066 ts_tablet_manager.cc:1939] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P 1f7bc557402b462d94288962ae3a2be5: Deleting consensus metadata
W20260620 18:44:25.097532 22762 connection.cc:441] server connection from 127.0.0.1:44436 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44436 (request call id 1) could send its response
I20260620 18:44:25.094055 23065 ts_tablet_manager.cc:1905] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:25.099582 23065 ts_tablet_manager.cc:1918] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:25.099666 23065 log.cc:1196] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/9f719b7d2f624ba4b92a69e2f8e01bf8
I20260620 18:44:25.099933 23065 ts_tablet_manager.cc:1939] T 9f719b7d2f624ba4b92a69e2f8e01bf8 P c37364286d744b818ba2ce19ac421134: Deleting consensus metadata
W20260620 18:44:25.100273 22628 connection.cc:441] server connection from 127.0.0.1:47878 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47878 (request call id 1) could send its response
I20260620 18:44:25.101302 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:25.085866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.15.77.124:39841)
0620 18:44:25.085943 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:25.085945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:25.085952 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:44:25.085971 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:25.085971 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:25.085982 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:25.086033 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:25.087439 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.088202 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.089317 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.089632 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.089675 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:25.091890 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:25.091892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:25.091892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:25.091900 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:25.094862 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:25.100411 (+ 5549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:25.100413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:25.100413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:25.100481 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:25.100704 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:25.100705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:25.100705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:25.100832 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:25.100918 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:25.101114 (+ 196us) server_negotiation.cc:299] Negotiation successful
0620 18:44:25.101204 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 18:44:25.103577 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:25.094731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.15.77.125:40663)
0620 18:44:25.094782 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:25.094784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:25.094914 (+ 130us) server_negotiation.cc:407] Connection header received
0620 18:44:25.095011 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:25.095012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:25.095027 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:25.095057 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:25.096100 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.096854 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.097882 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.098204 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.098242 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:25.099996 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:25.099998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:25.099998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:25.100006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:25.102005 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:25.102422 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:25.102423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:25.102423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:25.102479 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:25.102685 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:25.102686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:25.102686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:25.102844 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:25.102947 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:25.103379 (+ 432us) server_negotiation.cc:299] Negotiation successful
0620 18:44:25.103475 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:44:25.103605 22745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:25.094303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52618 (local address 127.15.77.124:39841)
0620 18:44:25.094364 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:25.094366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:25.094692 (+ 326us) server_negotiation.cc:407] Connection header received
0620 18:44:25.094799 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:25.094800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:25.094811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:25.094843 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:25.095818 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.096583 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.097883 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:25.098174 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:25.098215 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:25.100085 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:25.100086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:25.100086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:25.100093 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:25.102427 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:25.102809 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:25.102810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:25.102811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:25.102861 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:25.103066 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:25.103066 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:25.103067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:25.103208 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:25.103305 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:25.103454 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 18:44:25.103521 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 18:44:26.081215 23075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:26.080885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:46673)
0620 18:44:26.081069 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:26.081151 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
W20260620 18:44:26.081429 22876 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:38851 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:26.081431 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:26.081075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:40209)
0620 18:44:26.081190 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:26.081380 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":53,"threads_started":1}
W20260620 18:44:26.081616 23012 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:38851 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:26.081895 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:26.081574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:44673)
0620 18:44:26.081742 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:26.081844 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":51,"threads_started":1}
W20260620 18:44:26.082062 22739 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:38851 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:26.585508 23078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3afa68609e9b4416a496e310aba3a205)
I20260620 18:44:26.585605 23078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:26.585847 23078 leader_election.cc:288] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851)
I20260620 18:44:26.586270 23079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:26.586062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40672)
0620 18:44:26.586174 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:26.586232 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":68,"threads_started":1}
W20260620 18:44:26.586429 22359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:26.586848 23079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:26.586738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40686)
0620 18:44:26.586791 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:26.586824 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 18:44:26.586947 22359 leader_election.cc:334] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851): Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:26.587040 22359 leader_election.cc:302] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 56cb9590fadb4cf68fc41eced963ab04; no voters: 3afa68609e9b4416a496e310aba3a205
I20260620 18:44:26.587155 23078 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:44:27.082139 23082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:27.081929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:44147)
0620 18:44:27.082042 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:27.082077 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260620 18:44:27.082342 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:27.081865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:42717)
0620 18:44:27.082021 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:27.082282 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260620 18:44:27.082638 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:27.082364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:53793)
0620 18:44:27.082554 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:27.082605 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":47,"threads_started":1}
I20260620 18:44:28.083014 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:28.082796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:39673)
0620 18:44:28.082921 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:28.082961 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260620 18:44:28.083014 23084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:28.082789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:45615)
0620 18:44:28.082921 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:28.082961 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260620 18:44:28.083509 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:28.083135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:54149)
0620 18:44:28.083408 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:28.083447 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":49,"threads_started":1}
I20260620 18:44:28.374257 23087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3afa68609e9b4416a496e310aba3a205)
I20260620 18:44:28.374387 23087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:28.374590 23087 leader_election.cc:288] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851)
I20260620 18:44:28.375123 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:28.374900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40906)
0620 18:44:28.375044 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:28.375085 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260620 18:44:28.375607 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:28.375508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40912)
0620 18:44:28.375553 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:28.375588 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 18:44:28.375689 22359 leader_election.cc:334] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851): Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:28.375730 22359 leader_election.cc:302] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 56cb9590fadb4cf68fc41eced963ab04; no voters: 3afa68609e9b4416a496e310aba3a205
I20260620 18:44:28.375823 23087 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:44:29.083905 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:29.083661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:52017)
0620 18:44:29.083802 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:29.083843 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
I20260620 18:44:29.084044 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:29.083717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:54455)
0620 18:44:29.083964 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:29.084004 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":56,"threads_started":1}
I20260620 18:44:29.084503 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:29.083841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:59777)
0620 18:44:29.084403 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:29.084439 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":44,"threads_started":1}
I20260620 18:44:30.084867 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:30.084577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:41415)
0620 18:44:30.084761 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:30.084803 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":61,"threads_started":1}
I20260620 18:44:30.085148 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:30.084568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:40991)
0620 18:44:30.084707 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:30.085102 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260620 18:44:30.085229 23095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:30.084871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:38235)
0620 18:44:30.085150 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:30.085185 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":50,"threads_started":1}
I20260620 18:44:30.892207 23096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3afa68609e9b4416a496e310aba3a205)
I20260620 18:44:30.892345 23096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:30.892575 23096 leader_election.cc:288] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851)
I20260620 18:44:30.893075 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:30.892823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:41174)
0620 18:44:30.892931 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:30.893025 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":60,"threads_started":1}
I20260620 18:44:30.893482 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:30.893381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:41180)
0620 18:44:30.893427 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:30.893457 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 18:44:30.893591 22359 leader_election.cc:334] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851): Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:30.893661 22359 leader_election.cc:302] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 56cb9590fadb4cf68fc41eced963ab04; no voters: 3afa68609e9b4416a496e310aba3a205
I20260620 18:44:30.893733 23096 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:44:31.085858 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:31.085474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:50287)
0620 18:44:31.085743 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:31.085795 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":124,"threads_started":1}
I20260620 18:44:31.086081 23100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:31.085804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:45621)
0620 18:44:31.085995 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:31.086032 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":61,"threads_started":1}
I20260620 18:44:31.086206 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:31.085954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:56803)
0620 18:44:31.086134 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:31.086166 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":53,"threads_started":1}
I20260620 18:44:32.086848 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:32.086581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:51397)
0620 18:44:32.086732 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:32.086787 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260620 18:44:32.086922 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:32.086508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:33509)
0620 18:44:32.086825 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:32.086872 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":91,"threads_started":1}
I20260620 18:44:32.087262 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:32.086819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:56457)
0620 18:44:32.087177 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:32.087215 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":46,"threads_started":1}
I20260620 18:44:32.407867 23105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3afa68609e9b4416a496e310aba3a205)
I20260620 18:44:32.408142 23105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:32.408277 23105 leader_election.cc:288] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851)
I20260620 18:44:32.408694 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:32.408469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:47268)
0620 18:44:32.408604 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:32.408654 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
W20260620 18:44:32.408788 22359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260620 18:44:32.409222 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:32.409073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:47276)
0620 18:44:32.409173 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:32.409202 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260620 18:44:32.409332 22359 leader_election.cc:334] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851): Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:32.409399 22359 leader_election.cc:302] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 56cb9590fadb4cf68fc41eced963ab04; no voters: 3afa68609e9b4416a496e310aba3a205
I20260620 18:44:32.409525 23105 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:44:33.087828 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:33.087528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:60357)
0620 18:44:33.087692 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:33.087743 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
I20260620 18:44:33.088086 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:33.087888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:48727)
0620 18:44:33.088008 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:33.088044 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":49,"threads_started":1}
I20260620 18:44:33.088356 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:33.087783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:47717)
0620 18:44:33.088260 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:33.088300 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":53,"threads_started":1}
I20260620 18:44:34.088778 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:34.088522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:57793)
0620 18:44:34.088676 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:34.088718 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260620 18:44:34.089113 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:34.088923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:37907)
0620 18:44:34.089010 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:34.089048 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":45,"threads_started":1}
I20260620 18:44:34.089124 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:34.088894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:59831)
0620 18:44:34.089039 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:34.089077 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260620 18:44:35.089749 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:35.089340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:46055)
0620 18:44:35.089646 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:35.089691 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":63,"threads_started":1}
I20260620 18:44:35.089928 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:35.089748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:41639)
0620 18:44:35.089855 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:35.089884 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":49,"threads_started":1}
I20260620 18:44:35.089929 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:35.089739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:48037)
0620 18:44:35.089853 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:35.089885 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
I20260620 18:44:36.090672 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:36.090440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:56407)
0620 18:44:36.090571 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:36.090613 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260620 18:44:36.090672 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:36.090302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:44625)
0620 18:44:36.090567 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:36.090621 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":63,"threads_started":1}
I20260620 18:44:36.090726 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:36.090561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:37253)
0620 18:44:36.090657 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:36.090686 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":46,"threads_started":1}
I20260620 18:44:37.091423 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:37.091105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:35035)
0620 18:44:37.091295 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:37.091362 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":79,"threads_started":1}
I20260620 18:44:37.091481 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:37.091153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:54929)
0620 18:44:37.091396 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:37.091434 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":59,"threads_started":1}
I20260620 18:44:37.091675 23122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:37.091153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:35865)
0620 18:44:37.091598 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:37.091634 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":56,"threads_started":1}
I20260620 18:44:38.092406 23123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:38.092050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:56603)
0620 18:44:38.092296 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:38.092340 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":71,"threads_started":1}
I20260620 18:44:38.092420 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:38.092204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:38177)
0620 18:44:38.092327 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:38.092362 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
I20260620 18:44:38.092618 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:38.092256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:51605)
0620 18:44:38.092542 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:38.092579 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":49,"threads_started":1}
I20260620 18:44:39.093122 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:39.092833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:39853)
0620 18:44:39.093019 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:39.093064 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":63,"threads_started":1}
I20260620 18:44:39.093209 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:39.092992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:41045)
0620 18:44:39.093127 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:39.093162 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260620 18:44:39.093508 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:39.093084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:38109)
0620 18:44:39.093396 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:39.093465 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":70,"threads_started":1}
I20260620 18:44:39.911171 23129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3afa68609e9b4416a496e310aba3a205)
I20260620 18:44:39.911301 23129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:39.911514 23129 leader_election.cc:288] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851)
I20260620 18:44:39.911885 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:39.911702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:47748)
0620 18:44:39.911801 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:39.911847 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":59,"threads_started":1}
W20260620 18:44:39.911976 22359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:39.912344 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:39.912248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:47752)
0620 18:44:39.912299 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:39.912326 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 18:44:39.912426 22359 leader_election.cc:334] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851): Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:39.912482 22359 leader_election.cc:302] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 56cb9590fadb4cf68fc41eced963ab04; no voters: 3afa68609e9b4416a496e310aba3a205
I20260620 18:44:39.912567 23129 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:44:40.093961 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:40.093739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:53457)
0620 18:44:40.093874 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:40.093911 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":57,"threads_started":1}
I20260620 18:44:40.093961 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:40.093720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:51505)
0620 18:44:40.093874 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:40.093911 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260620 18:44:40.094380 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:40.093973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:49743)
0620 18:44:40.094313 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:40.094345 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":67,"threads_started":1}
I20260620 18:44:41.094756 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:41.094539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:34623)
0620 18:44:41.094667 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:41.094706 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260620 18:44:41.094756 23135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:41.094516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:41475)
0620 18:44:41.094667 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:41.094705 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260620 18:44:41.095032 23137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:41.094712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:36991)
0620 18:44:41.094928 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:41.094969 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":52,"threads_started":1}
I20260620 18:44:42.095516 23139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:42.095220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:43941)
0620 18:44:42.095399 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:42.095443 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":75,"threads_started":1}
I20260620 18:44:42.095589 23138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:42.095156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:52063)
0620 18:44:42.095506 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:42.095545 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":82,"threads_started":1}
I20260620 18:44:42.095986 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:42.095438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:47127)
0620 18:44:42.095904 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:42.095939 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":47,"threads_started":1}
I20260620 18:44:42.920486 23141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3afa68609e9b4416a496e310aba3a205)
I20260620 18:44:42.920641 23141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:42.920842 23141 leader_election.cc:288] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851)
I20260620 18:44:42.921205 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:42.921002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:39990)
0620 18:44:42.921114 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:42.921161 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260620 18:44:42.921888 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:42.921755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:39998)
0620 18:44:42.921826 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:42.921861 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260620 18:44:42.922019 22359 leader_election.cc:334] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851): Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:42.922089 22359 leader_election.cc:302] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 56cb9590fadb4cf68fc41eced963ab04; no voters: 3afa68609e9b4416a496e310aba3a205
I20260620 18:44:42.922181 23141 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:44:43.096395 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:43.096080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:46647)
0620 18:44:43.096295 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:43.096336 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":62,"threads_started":1}
I20260620 18:44:43.096441 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:43.096081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:39583)
0620 18:44:43.096352 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:43.096394 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":68,"threads_started":1}
I20260620 18:44:43.096692 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:43.096305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:59677)
0620 18:44:43.096601 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:43.096635 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":63,"threads_started":1}
I20260620 18:44:44.097277 23147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:44.096765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:54021)
0620 18:44:44.096932 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:44.097224 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260620 18:44:44.097275 23148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:44.097066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:48857)
0620 18:44:44.097177 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:44.097217 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
I20260620 18:44:44.097401 23149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:44.097103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:41121)
0620 18:44:44.097319 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:44.097356 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":54,"threads_started":1}
I20260620 18:44:45.097898 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:45.097668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:59979)
0620 18:44:45.097800 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:45.097840 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260620 18:44:45.097980 23151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:45.097779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:60625)
0620 18:44:45.097903 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:45.097940 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":54,"threads_started":1}
I20260620 18:44:45.098155 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:45.097779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:47355)
0620 18:44:45.098082 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:45.098115 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":57,"threads_started":1}
I20260620 18:44:45.700380 23153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3afa68609e9b4416a496e310aba3a205)
I20260620 18:44:45.700531 23153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:45.700721 23153 leader_election.cc:288] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851)
I20260620 18:44:45.701109 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:45.700919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40114)
0620 18:44:45.701023 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:45.701064 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":54,"threads_started":1}
W20260620 18:44:45.701195 22359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260620 18:44:45.701587 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:45.701489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:40122)
0620 18:44:45.701540 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:45.701568 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 18:44:45.701701 22359 leader_election.cc:334] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851): Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:45.701797 22359 leader_election.cc:302] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 56cb9590fadb4cf68fc41eced963ab04; no voters: 3afa68609e9b4416a496e310aba3a205
I20260620 18:44:45.701889 23153 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:44:46.098656 23157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:46.098389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:33753)
0620 18:44:46.098537 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:46.098599 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260620 18:44:46.098910 23158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:46.098677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:37769)
0620 18:44:46.098845 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:46.098875 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260620 18:44:46.098907 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:46.098391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:59083)
0620 18:44:46.098822 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:46.098861 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":77,"threads_started":1}
I20260620 18:44:47.099342 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:47.099086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:57971)
0620 18:44:47.099216 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:47.099256 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260620 18:44:47.099684 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:47.099413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:54101)
0620 18:44:47.099588 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:47.099646 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260620 18:44:47.099685 23160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:47.099393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:49403)
0620 18:44:47.099600 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:47.099640 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":67,"threads_started":1}
I20260620 18:44:48.100255 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:48.099976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:56425)
0620 18:44:48.100153 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:48.100196 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":67,"threads_started":1}
I20260620 18:44:48.100322 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:48.100119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:45587)
0620 18:44:48.100235 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:48.100263 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":54,"threads_started":1}
I20260620 18:44:48.100497 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:48.100277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:55615)
0620 18:44:48.100415 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:48.100451 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":51,"threads_started":1}
I20260620 18:44:49.101114 23165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:49.100828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:37115)
0620 18:44:49.100981 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:49.101021 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
I20260620 18:44:49.101166 23166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:49.100871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:43267)
0620 18:44:49.101065 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:49.101110 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":44,"threads_started":1}
I20260620 18:44:49.101188 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:49.101018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:52533)
0620 18:44:49.101117 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:49.101146 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":45,"threads_started":1}
I20260620 18:44:50.101961 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:50.101686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:53975)
0620 18:44:50.101855 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:50.101901 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260620 18:44:50.101966 23168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:50.101695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:56757)
0620 18:44:50.101872 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:50.101913 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":65,"threads_started":1}
I20260620 18:44:50.102065 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:50.101874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:35489)
0620 18:44:50.101979 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:50.102021 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
I20260620 18:44:51.102689 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:51.102392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:54619)
0620 18:44:51.102592 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:51.102633 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":69,"threads_started":1}
I20260620 18:44:51.102689 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:51.102439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:52953)
0620 18:44:51.102590 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:51.102632 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":58,"threads_started":1}
I20260620 18:44:51.102916 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:51.102392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:59885)
0620 18:44:51.102838 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:51.102875 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":112,"threads_started":1}
I20260620 18:44:52.103691 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:52.103242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:34577)
0620 18:44:52.103596 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:52.103642 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":97,"threads_started":1}
I20260620 18:44:52.103713 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:52.103389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:37879)
0620 18:44:52.103595 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:52.103637 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":53,"threads_started":1}
I20260620 18:44:52.103714 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:52.103453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:38543)
0620 18:44:52.103646 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:52.103678 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":62,"threads_started":1}
I20260620 18:44:52.980423 23177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3afa68609e9b4416a496e310aba3a205)
I20260620 18:44:52.980527 23177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3afa68609e9b4416a496e310aba3a205" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38851 } } peers { permanent_uuid: "56cb9590fadb4cf68fc41eced963ab04" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 40663 } attrs { promote: false } } peers { permanent_uuid: "5d433ee43d8b4dd58922cabbe3098519" member_type: NON_VOTER last_known_addr { host: "127.15.77.124" port: 39841 } attrs { promote: true } }
I20260620 18:44:52.980660 23177 leader_election.cc:288] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851)
I20260620 18:44:52.981068 23178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:52.980857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:43838)
0620 18:44:52.980981 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:52.981028 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
W20260620 18:44:52.981231 22359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:52.981599 23178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:52.981503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.0.0.1:43854)
0620 18:44:52.981552 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:52.981579 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 18:44:52.981709 22359 leader_election.cc:334] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3afa68609e9b4416a496e310aba3a205 (127.15.77.126:38851): Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
I20260620 18:44:52.981771 22359 leader_election.cc:302] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 56cb9590fadb4cf68fc41eced963ab04; no voters: 3afa68609e9b4416a496e310aba3a205
I20260620 18:44:52.981870 23177 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56cb9590fadb4cf68fc41eced963ab04 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:44:53.104597 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:53.104220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:43119)
0620 18:44:53.104495 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:53.104551 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":57,"threads_started":1}
I20260620 18:44:53.104597 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:53.104206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:34535)
0620 18:44:53.104496 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:53.104541 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":70,"threads_started":1}
I20260620 18:44:53.104754 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:53.104440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:60649)
0620 18:44:53.104597 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:53.104635 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":43,"threads_started":1}
I20260620 18:44:54.105341 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:54.105108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:35473)
0620 18:44:54.105220 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:54.105262 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":42,"threads_started":1}
I20260620 18:44:54.105391 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:54.105060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:47345)
0620 18:44:54.105315 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:54.105347 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":85,"threads_started":1}
I20260620 18:44:54.105629 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:54.105227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:45317)
0620 18:44:54.105529 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:54.105570 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":62,"threads_started":1}
I20260620 18:44:55.106256 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:55.106003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.66:59311)
0620 18:44:55.106153 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:55.106203 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":53,"threads_started":1}
I20260620 18:44:55.106339 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:55.106117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.65:58347)
0620 18:44:55.106214 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:55.106251 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":42,"threads_started":1}
I20260620 18:44:55.106482 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:55.105882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38851 (local address 127.15.77.67:54269)
0620 18:44:55.106246 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:55.106275 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":64,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.15.77.126:38851,127.15.77.125:40663,127.15.77.124:39841, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.15.77.126:38851,127.15.77.125:40663,127.15.77.124:39841, num_attempts: 1)
I20260620 18:44:55.286343 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 22519
I20260620 18:44:55.291550 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 22746
I20260620 18:44:55.296859 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 22882
I20260620 18:44:55.302286 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 22238
I20260620 18:44:55.307557 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 22426
2026-06-20T18:44:55Z chronyd exiting
I20260620 18:44:55.324133 15669 test_util.cc:175] -----------------------------------------------
I20260620 18:44:55.324195 15669 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781981032911509-15669-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (33198 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:44:55 dist-test-slave-tvl9 krb5kdc[23191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:44:55 dist-test-slave-tvl9 krb5kdc[23191](info): set up 2 sockets
Jun 20 18:44:55 dist-test-slave-tvl9 krb5kdc[23191](info): commencing operation
krb5kdc: starting...
W20260620 18:44:56.371814 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:44:56 dist-test-slave-tvl9 krb5kdc[23191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:44:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.126.keytab.
Entry for principal oryx/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.126.keytab.
I20260620 18:44:56.491564 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:41979
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.126.keytab
--principal=oryx/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:32871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:41979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:56.574416 23204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:56.574693 23204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:56.574730 23204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:56.575958 23204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:56.576036 23204 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 18:44:56.576058 23204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:56.576079 23204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:56.576112 23204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:56.576135 23204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:56.577749 23204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:32871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:41979
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.126.keytab
--principal=oryx/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:41979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:56.578065 23204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:56.578310 23204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:56.583868 23212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:56.583796 23209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:56.583778 23210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:56.583892 23204 server_base.cc:1034] running on GCE node
I20260620 18:44:56.584254 23204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:56.584524 23204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:56.585668 23204 hybrid_clock.cc:648] HybridClock initialized: now 1781981096585648 us; error 38 us; skew 500 ppm
Jun 20 18:44:56 dist-test-slave-tvl9 krb5kdc[23191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981096, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:56.588285 23204 init.cc:377] Logged in from keytab as oryx/127.15.77.126@KRBTEST.COM (short username oryx)
I20260620 18:44:56.588690 23204 webserver.cc:460] Webserver started at http://127.15.77.126:43871/ using document root <none> and password file <none>
I20260620 18:44:56.588943 23204 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:56.589015 23204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:56.589164 23204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:56.590252 23204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "794c62ba67e440d699607c9814312407"
format_stamp: "Formatted at 2026-06-20 18:44:56 on dist-test-slave-tvl9"
server_key: "97134f1ad39fd16fbb854e6b74307385"
server_key_iv: "b9df651fac728de9d1198af427bf342e"
server_key_version: "encryptionkey@0"
I20260620 18:44:56.590619 23204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "794c62ba67e440d699607c9814312407"
format_stamp: "Formatted at 2026-06-20 18:44:56 on dist-test-slave-tvl9"
server_key: "97134f1ad39fd16fbb854e6b74307385"
server_key_iv: "b9df651fac728de9d1198af427bf342e"
server_key_version: "encryptionkey@0"
I20260620 18:44:56.592072 23204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:56.592872 23219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:56.593045 23204 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:56.593168 23204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "794c62ba67e440d699607c9814312407"
format_stamp: "Formatted at 2026-06-20 18:44:56 on dist-test-slave-tvl9"
server_key: "97134f1ad39fd16fbb854e6b74307385"
server_key_iv: "b9df651fac728de9d1198af427bf342e"
server_key_version: "encryptionkey@0"
I20260620 18:44:56.593246 23204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:56.603896 23204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:56.605873 23204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:56.606079 23204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:56.610855 23204 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:41979
I20260620 18:44:56.610957 23281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:41979 every 8 connection(s)
I20260620 18:44:56.611279 23204 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:44:56.612093 23282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:56.614388 23282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407: Bootstrap starting.
I20260620 18:44:56.615049 23282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:56.615432 23282 log.cc:824] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:56.616130 23282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407: No bootstrap required, opened a new log
I20260620 18:44:56.616524 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 23204
I20260620 18:44:56.616623 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:44:56.616845 15669 external_mini_cluster.cc:1428] Setting key bd396530f9b5fb4591af64415e1a59af
I20260620 18:44:56.618827 23282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794c62ba67e440d699607c9814312407" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41979 } }
I20260620 18:44:56.619012 23282 raft_consensus.cc:374] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:56.619056 23282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 794c62ba67e440d699607c9814312407, State: Initialized, Role: FOLLOWER
I20260620 18:44:56.619293 23282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794c62ba67e440d699607c9814312407" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41979 } }
I20260620 18:44:56.619421 23282 raft_consensus.cc:388] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:44:56.619493 23282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:44:56.619554 23282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:56.620325 23282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794c62ba67e440d699607c9814312407" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41979 } }
I20260620 18:44:56.620455 23282 leader_election.cc:302] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 794c62ba67e440d699607c9814312407; no voters:
I20260620 18:44:56.620661 23282 leader_election.cc:288] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:44:56.620705 23287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:56.620862 23287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [term 1 LEADER]: Becoming Leader. State: Replica: 794c62ba67e440d699607c9814312407, State: Running, Role: LEADER
I20260620 18:44:56.620957 23282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:44:56.621069 23287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794c62ba67e440d699607c9814312407" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41979 } }
I20260620 18:44:56.621447 23289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "794c62ba67e440d699607c9814312407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794c62ba67e440d699607c9814312407" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41979 } } }
I20260620 18:44:56.621659 23289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:56.621439 23287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 794c62ba67e440d699607c9814312407. Latest consensus state: current_term: 1 leader_uuid: "794c62ba67e440d699607c9814312407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "794c62ba67e440d699607c9814312407" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41979 } } }
I20260620 18:44:56.621881 23287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407 [sys.catalog]: This master's current role is: LEADER
I20260620 18:44:56.622058 23296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:44:56.622654 23296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:44:56.624445 23296 catalog_manager.cc:1347] Generated new cluster ID: df30cf24c6cd4f289122ff6d9ed6db63
I20260620 18:44:56.624498 23296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:44:56 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:56.630146 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:56.617386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.15.77.126:41979)
0620 18:44:56.617565 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:56.617574 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:56.617607 (+ 33us) server_negotiation.cc:407] Connection header received
0620 18:44:56.618797 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:56.618804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:56.618860 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:56.618983 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:56.619874 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.621258 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.622417 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.623013 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.623134 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:56.626348 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:56.626352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:56.626353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:56.626366 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:56.628451 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:56.628885 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:56.628888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:56.628888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:56.629011 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:56.629236 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:56.629236 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:56.629237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:56.629532 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:56.629685 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:56.629810 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 18:44:56.629943 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260620 18:44:56.632781 23296 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:44:56.633277 23296 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:44:56.642971 23296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407: Generated new TSK 0
I20260620 18:44:56.643211 23296 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab.
Entry for principal oryx/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab.
I20260620 18:44:56.672451 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41979
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab
--principal=oryx/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:32871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:56.762300 23308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:56.762550 23308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:56.762594 23308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:56.763908 23308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:56.763972 23308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:56.764010 23308 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 18:44:56.764068 23308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:44:56.765637 23308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:32871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab
--principal=oryx/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.77.126:41979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:56.765916 23308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:56.766160 23308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:56.772136 23316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:56.772182 23313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:56.772136 23314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:56.772423 23308 server_base.cc:1034] running on GCE node
I20260620 18:44:56.772655 23308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:56.772876 23308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:56.774052 23308 hybrid_clock.cc:648] HybridClock initialized: now 1781981096774041 us; error 40 us; skew 500 ppm
Jun 20 18:44:56 dist-test-slave-tvl9 krb5kdc[23191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981096, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:56.776619 23308 init.cc:377] Logged in from keytab as oryx/127.15.77.65@KRBTEST.COM (short username oryx)
I20260620 18:44:56.777103 23308 webserver.cc:460] Webserver started at http://127.15.77.65:43987/ using document root <none> and password file <none>
I20260620 18:44:56.777343 23308 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:56.777412 23308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:56.777573 23308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:56.778699 23308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a"
format_stamp: "Formatted at 2026-06-20 18:44:56 on dist-test-slave-tvl9"
server_key: "53a13861081c386a99c2c5942720579c"
server_key_iv: "58e17f7ff87d6740d486673ab1b98e1a"
server_key_version: "encryptionkey@0"
I20260620 18:44:56.779076 23308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a"
format_stamp: "Formatted at 2026-06-20 18:44:56 on dist-test-slave-tvl9"
server_key: "53a13861081c386a99c2c5942720579c"
server_key_iv: "58e17f7ff87d6740d486673ab1b98e1a"
server_key_version: "encryptionkey@0"
I20260620 18:44:56.780679 23308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:56.781507 23323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:56.781700 23308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:56.781802 23308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a"
format_stamp: "Formatted at 2026-06-20 18:44:56 on dist-test-slave-tvl9"
server_key: "53a13861081c386a99c2c5942720579c"
server_key_iv: "58e17f7ff87d6740d486673ab1b98e1a"
server_key_version: "encryptionkey@0"
I20260620 18:44:56.781890 23308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:56.805127 23308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:56.807368 23308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:56.807554 23308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:56.808183 23308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:56.808262 23308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:56.808307 23308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:56.808334 23308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 18:44:56 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981096, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:56.816035 23308 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:33647
I20260620 18:44:56.816157 23438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:33647 every 8 connection(s)
I20260620 18:44:56.816510 23308 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:44:56.818461 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 23308
I20260620 18:44:56.818568 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:44:56.818791 15669 external_mini_cluster.cc:1428] Setting key 798b124b22361240b3e8efbe0d0a7db6
I20260620 18:44:56.818893 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:56.808762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:57833 (local address 127.15.77.126:41979)
0620 18:44:56.808823 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:56.808825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:56.809066 (+ 241us) server_negotiation.cc:407] Connection header received
0620 18:44:56.809565 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:56.809566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:56.809585 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:56.809661 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:56.810675 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.811579 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.812627 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.813001 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.813098 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:56.815827 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:56.815830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:56.815830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:56.815840 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:56.817478 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:56.817861 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:56.817862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:56.817862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:56.817917 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:56.818193 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:56.818194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:56.818194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:56.818321 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:56.818373 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:56.818753 (+ 380us) server_negotiation.cc:299] Negotiation successful
0620 18:44:56.818829 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 18:44:56.819186 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:56.808744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41979 (local address 127.15.77.65:57833)
0620 18:44:56.808935 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:56.808973 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:56.809457 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:56.809796 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:56.809799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:56.810220 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:56.810553 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:56.810560 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.811688 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.811689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:56.812529 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:56.812531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.813142 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.813143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:56.813325 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:56.813926 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:56.813938 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:56.815716 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:56.817554 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:56.817559 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:56.817561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:56.817788 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:56.818004 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:56.818005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:56.818005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:56.818124 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:56.818395 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:56.818403 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:56.818597 (+ 194us) client_negotiation.cc:769] Sending connection context
0620 18:44:56.818737 (+ 140us) client_negotiation.cc:240] Negotiation successful
0620 18:44:56.818924 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260620 18:44:56.819774 23439 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41979
I20260620 18:44:56.819911 23439 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:56.820148 23439 heartbeater.cc:511] Master 127.15.77.126:41979 requested a full tablet report, sending...
I20260620 18:44:56.820804 23236 ts_manager.cc:194] Registered new tserver with Master: bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647)
I20260620 18:44:56.821678 23236 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:57833
I20260620 18:44:56.827805 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:56.820925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:50215 (local address 127.15.77.126:41979)
0620 18:44:56.820970 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:56.820973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:56.821129 (+ 156us) server_negotiation.cc:407] Connection header received
0620 18:44:56.821203 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:56.821204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:56.821214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:56.821259 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:56.821713 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.822459 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.823592 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.823785 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.823852 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:56.824988 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:56.824990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:56.824990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:56.824999 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:56.826542 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:56.826956 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:56.826957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:56.826958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:56.827008 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:56.827231 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:56.827231 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:56.827232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:56.827411 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:56.827478 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:56.827660 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 18:44:56.827730 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:44:56.827888 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:56.820886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41979 (local address 127.15.77.65:50215)
0620 18:44:56.821043 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:56.821054 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:56.821144 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:56.821261 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:56.821262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:56.821503 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:56.821605 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:56.821607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.822594 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.822596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:56.823487 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:56.823491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.823871 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.823872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:56.823949 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:56.824411 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:56.824417 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:56.824879 (+ 462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:56.826633 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:56.826635 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:56.826636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:56.826878 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:56.827076 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:56.827077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:56.827077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:56.827158 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:56.827484 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:56.827485 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:56.827548 (+ 63us) client_negotiation.cc:769] Sending connection context
0620 18:44:56.827640 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 18:44:56.827761 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for oryx/127.15.77.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab.
Entry for principal oryx/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab.
I20260620 18:44:56.847108 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41979
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab
--principal=oryx/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:32871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:56.933013 23448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:56.933264 23448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:56.933310 23448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:56.934515 23448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:56.934587 23448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:56.934623 23448 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 18:44:56.934672 23448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:44:56.936368 23448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:32871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab
--principal=oryx/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.77.126:41979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:56.936656 23448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:56.936913 23448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:56.942467 23456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:56.942508 23448 server_base.cc:1034] running on GCE node
W20260620 18:44:56.942541 23454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:56.942471 23453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:56.942931 23448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:56.943202 23448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:56.944388 23448 hybrid_clock.cc:648] HybridClock initialized: now 1781981096944379 us; error 38 us; skew 500 ppm
Jun 20 18:44:56 dist-test-slave-tvl9 krb5kdc[23191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981096, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:56.947032 23448 init.cc:377] Logged in from keytab as oryx/127.15.77.66@KRBTEST.COM (short username oryx)
I20260620 18:44:56.947604 23448 webserver.cc:460] Webserver started at http://127.15.77.66:41867/ using document root <none> and password file <none>
I20260620 18:44:56.947872 23448 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:56.947944 23448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:56.948101 23448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:56.949219 23448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "f30f957fba8444739c7c1f7b3fe41f02"
format_stamp: "Formatted at 2026-06-20 18:44:56 on dist-test-slave-tvl9"
server_key: "3945f44cf4ee065d1ee770f154b09336"
server_key_iv: "4aac1887ceb52b1b9274e765fbf62a06"
server_key_version: "encryptionkey@0"
I20260620 18:44:56.949661 23448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "f30f957fba8444739c7c1f7b3fe41f02"
format_stamp: "Formatted at 2026-06-20 18:44:56 on dist-test-slave-tvl9"
server_key: "3945f44cf4ee065d1ee770f154b09336"
server_key_iv: "4aac1887ceb52b1b9274e765fbf62a06"
server_key_version: "encryptionkey@0"
I20260620 18:44:56.951119 23448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:44:56.952045 23463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:56.952232 23448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:56.952330 23448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "f30f957fba8444739c7c1f7b3fe41f02"
format_stamp: "Formatted at 2026-06-20 18:44:56 on dist-test-slave-tvl9"
server_key: "3945f44cf4ee065d1ee770f154b09336"
server_key_iv: "4aac1887ceb52b1b9274e765fbf62a06"
server_key_version: "encryptionkey@0"
I20260620 18:44:56.952418 23448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:56.977377 23448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:56.979493 23448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:56.979663 23448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:56.980281 23448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:56.980350 23448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:56.980396 23448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:56.980420 23448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 18:44:56 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981096, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:56.987941 23448 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:37863
I20260620 18:44:56.988015 23578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:37863 every 8 connection(s)
I20260620 18:44:56.988329 23448 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:44:56.991012 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:56.980820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:46977 (local address 127.15.77.126:41979)
0620 18:44:56.980896 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:56.980900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:56.981101 (+ 201us) server_negotiation.cc:407] Connection header received
0620 18:44:56.981596 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:56.981597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:56.981621 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:56.981704 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:56.982667 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.983547 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.984678 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.985025 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.985118 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:56.987958 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:56.987960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:56.987960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:56.987967 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:56.989586 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:56.990015 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:56.990017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:56.990017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:56.990083 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:56.990336 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:56.990337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:56.990337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:56.990468 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:56.990541 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:56.990838 (+ 297us) server_negotiation.cc:299] Negotiation successful
0620 18:44:56.990917 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 18:44:56.991212 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:56.980789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41979 (local address 127.15.77.66:46977)
0620 18:44:56.980961 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:56.981005 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:56.981498 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:56.981814 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:56.981819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:56.982218 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:56.982555 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:56.982561 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.983651 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.983651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:56.984610 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:56.984613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.985137 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.985138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:56.985283 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:56.985879 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:56.985889 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:56.987850 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:56.989690 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:56.989697 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:56.989700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:56.989926 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:56.990148 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:56.990149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:56.990150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:56.990265 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:56.990547 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:56.990553 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:56.990701 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 18:44:56.990815 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 18:44:56.990980 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
I20260620 18:44:56.991753 23579 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41979
I20260620 18:44:56.991845 23579 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:56.992053 23579 heartbeater.cc:511] Master 127.15.77.126:41979 requested a full tablet report, sending...
I20260620 18:44:56.992493 23236 ts_manager.cc:194] Registered new tserver with Master: f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863)
I20260620 18:44:56.992667 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 23448
I20260620 18:44:56.992781 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:44:56.992995 15669 external_mini_cluster.cc:1428] Setting key 136fde66dec42c7734cd5adb7e9ab91c
I20260620 18:44:56.993037 23236 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:46977
I20260620 18:44:56.998979 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:56.993215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:40059 (local address 127.15.77.126:41979)
0620 18:44:56.993274 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:56.993276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:56.993283 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:44:56.993301 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:56.993302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:56.993311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:56.993363 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:56.993779 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.994306 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.995002 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.995214 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.995298 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:56.996324 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:56.996325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:56.996325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:56.996332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:56.997820 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:56.998162 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:56.998163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:56.998163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:56.998222 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:56.998419 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:56.998420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:56.998421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:56.998573 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:56.998653 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:56.998804 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 18:44:56.998883 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 18:44:56.998991 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:56.992822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41979 (local address 127.15.77.66:40059)
0620 18:44:56.992982 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:56.992994 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:56.993046 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:56.993396 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:56.993397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:56.993623 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:56.993707 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:56.993710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.994405 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.994406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:56.994916 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:56.994918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:56.995301 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:56.995302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:56.995393 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:56.995906 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:56.995912 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:56.996215 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:56.997904 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:56.997905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:56.997906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:56.998080 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:56.998286 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:56.998287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:56.998287 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:56.998348 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:56.998645 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:56.998645 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:56.998694 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 18:44:56.998779 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 18:44:56.998884 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for oryx/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab.
Entry for principal oryx/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab.
I20260620 18:44:57.020186 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41979
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab
--principal=oryx/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:32871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:57.109661 23588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:57.109867 23588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:57.109892 23588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:57.111006 23588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:57.111045 23588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:57.111066 23588 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 18:44:57.111084 23588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:44:57.112623 23588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:32871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab
--principal=oryx/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.77.126:41979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:57.112821 23588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:57.113005 23588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:44:57.118167 23594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:57.118185 23596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:57.118247 23588 server_base.cc:1034] running on GCE node
W20260620 18:44:57.118189 23593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:57.118616 23588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:57.118875 23588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:57.120060 23588 hybrid_clock.cc:648] HybridClock initialized: now 1781981097120032 us; error 60 us; skew 500 ppm
Jun 20 18:44:57 dist-test-slave-tvl9 krb5kdc[23191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981097, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:44:57.123055 23588 init.cc:377] Logged in from keytab as oryx/127.15.77.67@KRBTEST.COM (short username oryx)
I20260620 18:44:57.124567 23588 webserver.cc:460] Webserver started at http://127.15.77.67:45965/ using document root <none> and password file <none>
I20260620 18:44:57.124920 23588 fs_manager.cc:359] Metadata directory not provided
I20260620 18:44:57.125030 23588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:44:57.125202 23588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:44:57.127082 23588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "4109ccbef0b043068afd24ef7bdec85e"
format_stamp: "Formatted at 2026-06-20 18:44:57 on dist-test-slave-tvl9"
server_key: "9e50b60fec3be801c1e21bd8fe51e8c1"
server_key_iv: "0f56d3b337c3e7588a51fc2b0c57a896"
server_key_version: "encryptionkey@0"
I20260620 18:44:57.127633 23588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "4109ccbef0b043068afd24ef7bdec85e"
format_stamp: "Formatted at 2026-06-20 18:44:57 on dist-test-slave-tvl9"
server_key: "9e50b60fec3be801c1e21bd8fe51e8c1"
server_key_iv: "0f56d3b337c3e7588a51fc2b0c57a896"
server_key_version: "encryptionkey@0"
I20260620 18:44:57.129616 23588 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260620 18:44:57.130635 23603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:57.130858 23588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:57.130960 23588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "4109ccbef0b043068afd24ef7bdec85e"
format_stamp: "Formatted at 2026-06-20 18:44:57 on dist-test-slave-tvl9"
server_key: "9e50b60fec3be801c1e21bd8fe51e8c1"
server_key_iv: "0f56d3b337c3e7588a51fc2b0c57a896"
server_key_version: "encryptionkey@0"
I20260620 18:44:57.131048 23588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:44:57.149498 23588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:44:57.152218 23588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:44:57.152385 23588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:44:57.153110 23588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:44:57.153211 23588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:57.153260 23588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:44:57.153285 23588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 18:44:57 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981097, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:57.161345 23588 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:39975
I20260620 18:44:57.162083 23718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:39975 every 8 connection(s)
I20260620 18:44:57.162573 23588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:44:57.163734 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.153470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:59745 (local address 127.15.77.126:41979)
0620 18:44:57.153517 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.153520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.153767 (+ 247us) server_negotiation.cc:407] Connection header received
0620 18:44:57.154304 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.154306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.154319 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.154377 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:57.155165 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.155993 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.157072 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.157372 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.157436 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.160479 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:57.160481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:57.160481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:57.160489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:57.162077 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:57.162571 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:57.162572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:57.162573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:57.162648 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:57.162973 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:57.162974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:57.162975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:57.163147 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:57.163222 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.163549 (+ 327us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.163639 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:44:57.163944 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.153480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41979 (local address 127.15.77.67:59745)
0620 18:44:57.153632 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:57.153683 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:57.154191 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:57.154454 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:57.154457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:57.154763 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:57.155035 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.155042 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.156085 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.156086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.156978 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.156980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.157508 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.157510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.157660 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.158220 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:57.158230 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:57.160377 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:57.162175 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:57.162183 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:57.162185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:57.162473 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:57.162751 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:57.162753 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:57.162753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:57.162898 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:57.163219 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:57.163222 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:57.163383 (+ 161us) client_negotiation.cc:769] Sending connection context
0620 18:44:57.163522 (+ 139us) client_negotiation.cc:240] Negotiation successful
0620 18:44:57.163711 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260620 18:44:57.164427 23719 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41979
I20260620 18:44:57.164529 23719 heartbeater.cc:464] Registering TS with master...
I20260620 18:44:57.164770 23719 heartbeater.cc:511] Master 127.15.77.126:41979 requested a full tablet report, sending...
I20260620 18:44:57.165180 23236 ts_manager.cc:194] Registered new tserver with Master: 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975)
I20260620 18:44:57.165604 23236 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:59745
I20260620 18:44:57.166126 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 23588
I20260620 18:44:57.166231 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:44:57.166422 15669 external_mini_cluster.cc:1428] Setting key b47a9c25c611c22bebc831f2d47bc2eb
I20260620 18:44:57.167205 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:44:57.173039 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.165507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:49513 (local address 127.15.77.126:41979)
0620 18:44:57.165576 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.165578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.165781 (+ 203us) server_negotiation.cc:407] Connection header received
0620 18:44:57.165883 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.165884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.165895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.165954 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:57.167033 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.167907 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.169130 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.169316 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.169374 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.170468 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:57.170469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:57.170470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:57.170475 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:57.171887 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:57.172271 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:57.172272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:57.172273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:57.172334 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:57.172531 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:57.172532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:57.172532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:57.172640 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:57.172717 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.172868 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.172946 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 18:44:57.173043 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.165558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41979 (local address 127.15.77.67:49513)
0620 18:44:57.165716 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:57.165727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:57.165808 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:57.166606 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:57.166608 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:57.166838 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:57.166908 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.166909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.168037 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.168038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.169040 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.169043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.169395 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.169396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.169476 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.169923 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:57.169928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:57.170381 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:57.171995 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:57.171997 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:57.171997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:57.172185 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:57.172401 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:57.172402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:57.172402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:57.172460 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:57.172708 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:57.172709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:57.172755 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 18:44:57.172844 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 18:44:57.172947 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
Jun 20 18:44:57 dist-test-slave-tvl9 krb5kdc[23191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981097, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 18:44:57 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:44:57 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 18:44:57.198995 23286 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:44:57.192022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60944 (local address 127.15.77.126:41979)
0620 18:44:57.192080 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.192082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.192162 (+ 80us) server_negotiation.cc:407] Connection header received
0620 18:44:57.192248 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.192248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.192257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.192311 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:57.193139 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.193626 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.194914 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.195111 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.195173 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.198897 (+ 3724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60944: BlockingRecv error: recv got EOF from 127.0.0.1:60944 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Jun 20 18:44:57 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981097, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:57.206879 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.199937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60946 (local address 127.15.77.126:41979)
0620 18:44:57.199991 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.199992 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.200101 (+ 109us) server_negotiation.cc:407] Connection header received
0620 18:44:57.200188 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.200189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.200199 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.200247 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:57.200717 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.201193 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.201883 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.202074 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.202152 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.204277 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:57.204278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:57.204279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:57.204287 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:57.205693 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:57.206084 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:57.206085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:57.206085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:57.206146 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:57.206358 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:57.206359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:57.206360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:57.206479 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:57.206586 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.206693 (+ 107us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.206773 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 18:44:57.208626 23236 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:44:57.210216 23236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:57.218004 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.212889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:37863 (local address 127.0.0.1:53308)
0620 18:44:57.213574 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:57.213583 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:57.213620 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:57.213836 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:57.213836 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:57.213838 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:57.213866 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.213867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.215183 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.215184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.216629 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.216632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.217678 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.217679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.217768 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.217787 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 18:44:57.217853 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 18:44:57.217912 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":66,"thread_start_us":99,"threads_started":1}
I20260620 18:44:57.218182 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.212682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33647 (local address 127.0.0.1:43476)
0620 18:44:57.213414 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:57.213427 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:57.213494 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:57.213743 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:57.213750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:57.213753 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:57.213837 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.213840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.215343 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.215344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.216727 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.216731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.217869 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.217870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.217959 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.217965 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 18:44:57.218009 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 18:44:57.218063 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":5,"thread_start_us":79,"threads_started":1}
I20260620 18:44:57.218381 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.212846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46704 (local address 127.15.77.67:39975)
0620 18:44:57.213023 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.213029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.213213 (+ 184us) server_negotiation.cc:407] Connection header received
0620 18:44:57.213370 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.213372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.213391 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.213463 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:57.213943 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.215074 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.216736 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.217800 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.217860 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.217940 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.218224 (+ 284us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.218264 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260620 18:44:57.218564 23744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.212870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43476 (local address 127.15.77.65:33647)
0620 18:44:57.213043 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.213047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.213593 (+ 546us) server_negotiation.cc:407] Connection header received
0620 18:44:57.213637 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.213640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.213656 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.213748 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:57.213929 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.215212 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.216846 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.217761 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.218275 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.218366 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.218394 (+ 28us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.218442 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
I20260620 18:44:57.218744 23742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.212681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:39975 (local address 127.0.0.1:46704)
0620 18:44:57.213088 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:57.213135 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:57.213230 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:57.213544 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:57.213804 (+ 260us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:57.213808 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:57.213859 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.213861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.215178 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.215486 (+ 308us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.216628 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.216632 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.217972 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.217973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.218030 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.218034 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:44:57.218084 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 18:44:57.218133 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":1,"thread_start_us":56,"threads_started":1}
I20260620 18:44:57.218811 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.213242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53308 (local address 127.15.77.66:37863)
0620 18:44:57.213635 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.213640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.213649 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:44:57.213685 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.213689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.213704 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.213771 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:57.213941 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.215074 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.216734 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.217483 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.217554 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.217646 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.218672 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.218725 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":81,"threads_started":1}
I20260620 18:44:57.219556 23513 tablet_service.cc:1467] Processing CreateTablet for tablet c30730cf28a14d01b6bd18a6001431f6 (DEFAULT_TABLE table=test-table [id=bd5e0bb9956c4aba9c509994e1b51b3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:57.219673 23634 tablet_service.cc:1467] Processing CreateTablet for tablet c30730cf28a14d01b6bd18a6001431f6 (DEFAULT_TABLE table=test-table [id=bd5e0bb9956c4aba9c509994e1b51b3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:57.219987 23513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c30730cf28a14d01b6bd18a6001431f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:57.220053 23634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c30730cf28a14d01b6bd18a6001431f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:57.225189 23747 tablet_bootstrap.cc:492] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: Bootstrap starting.
I20260620 18:44:57.225929 23747 tablet_bootstrap.cc:654] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:57.226333 23747 log.cc:824] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:57.227027 23749 tablet_bootstrap.cc:492] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: Bootstrap starting.
I20260620 18:44:57.227867 23749 tablet_bootstrap.cc:654] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:57.227854 23373 tablet_service.cc:1467] Processing CreateTablet for tablet c30730cf28a14d01b6bd18a6001431f6 (DEFAULT_TABLE table=test-table [id=bd5e0bb9956c4aba9c509994e1b51b3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:57.228250 23373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c30730cf28a14d01b6bd18a6001431f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:57.228675 23747 tablet_bootstrap.cc:492] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: No bootstrap required, opened a new log
I20260620 18:44:57.228756 23747 ts_tablet_manager.cc:1397] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 18:44:57.228886 23749 log.cc:824] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:57.230484 23749 tablet_bootstrap.cc:492] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: No bootstrap required, opened a new log
I20260620 18:44:57.230647 23749 ts_tablet_manager.cc:1397] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 18:44:57.230880 23747 raft_consensus.cc:348] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } }
I20260620 18:44:57.231112 23747 raft_consensus.cc:374] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:57.231161 23747 raft_consensus.cc:729] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30f957fba8444739c7c1f7b3fe41f02, State: Initialized, Role: FOLLOWER
I20260620 18:44:57.231297 23747 consensus_queue.cc:260] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } }
I20260620 18:44:57.231705 23747 ts_tablet_manager.cc:1428] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 18:44:57.231863 23579 heartbeater.cc:503] Master 127.15.77.126:41979 was elected leader, sending a full tablet report...
I20260620 18:44:57.232220 23752 tablet_bootstrap.cc:492] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: Bootstrap starting.
I20260620 18:44:57.232852 23749 raft_consensus.cc:348] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } }
I20260620 18:44:57.233003 23752 tablet_bootstrap.cc:654] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:57.233032 23749 raft_consensus.cc:374] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:57.233078 23749 raft_consensus.cc:729] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4109ccbef0b043068afd24ef7bdec85e, State: Initialized, Role: FOLLOWER
I20260620 18:44:57.233202 23749 consensus_queue.cc:260] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } }
I20260620 18:44:57.233355 23752 log.cc:824] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: Log is configured to *not* fsync() on all Append() calls
I20260620 18:44:57.233497 23749 ts_tablet_manager.cc:1428] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:44:57.233530 23719 heartbeater.cc:503] Master 127.15.77.126:41979 was elected leader, sending a full tablet report...
I20260620 18:44:57.234194 23752 tablet_bootstrap.cc:492] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: No bootstrap required, opened a new log
I20260620 18:44:57.234267 23752 ts_tablet_manager.cc:1397] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:44:57.235855 23752 raft_consensus.cc:348] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } }
I20260620 18:44:57.236002 23752 raft_consensus.cc:374] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:57.236050 23752 raft_consensus.cc:729] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfd69d052cf428ea35e6e4b4d7abb7a, State: Initialized, Role: FOLLOWER
I20260620 18:44:57.236166 23752 consensus_queue.cc:260] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } }
I20260620 18:44:57.236431 23752 ts_tablet_manager.cc:1428] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 18:44:57.236500 23439 heartbeater.cc:503] Master 127.15.77.126:41979 was elected leader, sending a full tablet report...
W20260620 18:44:57.238951 23580 tablet.cc:2369] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 18:44:57.318078 23440 tablet.cc:2369] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:44:57.398044 23751 raft_consensus.cc:482] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:57.398249 23751 raft_consensus.cc:504] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } }
I20260620 18:44:57.398665 23751 leader_election.cc:288] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975), bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647)
I20260620 18:44:57.399078 23753 raft_consensus.cc:482] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:57.399230 23753 raft_consensus.cc:504] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } }
I20260620 18:44:57.399686 23753 leader_election.cc:288] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647), f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863)
I20260620 18:44:57.402436 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.398864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:39975 (local address 127.15.77.66:55719)
0620 18:44:57.398917 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:57.398930 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:57.398978 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:57.399197 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:57.399198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:57.399200 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:57.399275 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.399277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.400248 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.400249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.401481 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.401483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.402185 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.402186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.402272 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.402275 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:44:57.402323 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 18:44:57.402361 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 18:44:57.402786 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.398937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33647 (local address 127.15.77.66:54269)
0620 18:44:57.399127 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:57.399137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:57.399191 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:57.399378 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:57.399379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:57.399381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:57.399822 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.399825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.400684 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.400685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.401801 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.401804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.402531 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.402532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.402614 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.402620 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 18:44:57.402668 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 18:44:57.402706 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":115,"threads_started":1}
I20260620 18:44:57.402874 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.398966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:55719 (local address 127.15.77.67:39975)
0620 18:44:57.399011 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.399014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.399023 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:44:57.399054 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.399054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.399065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.399127 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:57.399416 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.400146 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.401570 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.402060 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.402136 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.402145 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.402774 (+ 629us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.402812 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:44:57.403111 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c30730cf28a14d01b6bd18a6001431f6" candidate_uuid: "f30f957fba8444739c7c1f7b3fe41f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109ccbef0b043068afd24ef7bdec85e" is_pre_election: true
I20260620 18:44:57.403226 23673 raft_consensus.cc:2454] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30f957fba8444739c7c1f7b3fe41f02 in term 0.
I20260620 18:44:57.403501 23466 leader_election.cc:302] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4109ccbef0b043068afd24ef7bdec85e, f30f957fba8444739c7c1f7b3fe41f02; no voters:
I20260620 18:44:57.403594 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.399845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33647 (local address 127.15.77.67:52507)
0620 18:44:57.400046 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:57.400055 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:57.400089 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:57.400370 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:57.400371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:57.400373 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:57.400428 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.400429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.401333 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.401334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.402557 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.402560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.403295 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.403296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.403408 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.403412 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:44:57.403463 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 18:44:57.403511 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":39}
I20260620 18:44:57.403640 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.400026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:51685 (local address 127.15.77.66:37863)
0620 18:44:57.400061 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.400063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.400262 (+ 199us) server_negotiation.cc:407] Connection header received
0620 18:44:57.400348 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.400348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.400356 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.400383 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:57.400582 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.401269 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.402683 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.403235 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.403328 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.403336 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.403547 (+ 211us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.403583 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":172,"server-negotiator.queue_time_us":19}
I20260620 18:44:57.403640 23744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.399098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:54269 (local address 127.15.77.65:33647)
0620 18:44:57.399143 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.399146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.399239 (+ 93us) server_negotiation.cc:407] Connection header received
0620 18:44:57.399266 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.399267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.399280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.399374 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:57.399916 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.400600 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.401892 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.402425 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.402490 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.402498 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.402989 (+ 491us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.403575 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:44:57.403784 23751 raft_consensus.cc:2781] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:57.403846 23751 raft_consensus.cc:482] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:57.403874 23751 raft_consensus.cc:3037] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:57.403916 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.399915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:37863 (local address 127.15.77.67:51685)
0620 18:44:57.400209 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:57.400219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:57.400284 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:57.400424 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:57.400425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:57.400426 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:57.400498 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.400500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.401358 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.401359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.402569 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.402572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.403366 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.403367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.403465 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.403469 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:44:57.403797 (+ 328us) client_negotiation.cc:240] Negotiation successful
0620 18:44:57.403841 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":73,"threads_started":1}
I20260620 18:44:57.403976 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.400131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:52507 (local address 127.15.77.65:33647)
0620 18:44:57.400235 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.400237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.400244 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:44:57.400265 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.400266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.400274 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.400499 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:57.400539 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.401236 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.402641 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.403183 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.403289 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.403298 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.403670 (+ 372us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.403733 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":61,"threads_started":1}
I20260620 18:44:57.404057 23393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c30730cf28a14d01b6bd18a6001431f6" candidate_uuid: "f30f957fba8444739c7c1f7b3fe41f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" is_pre_election: true
I20260620 18:44:57.404210 23392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c30730cf28a14d01b6bd18a6001431f6" candidate_uuid: "4109ccbef0b043068afd24ef7bdec85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" is_pre_election: true
I20260620 18:44:57.404211 23393 raft_consensus.cc:2454] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30f957fba8444739c7c1f7b3fe41f02 in term 0.
I20260620 18:44:57.404398 23533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c30730cf28a14d01b6bd18a6001431f6" candidate_uuid: "4109ccbef0b043068afd24ef7bdec85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30f957fba8444739c7c1f7b3fe41f02" is_pre_election: true
I20260620 18:44:57.404872 23751 raft_consensus.cc:504] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } }
I20260620 18:44:57.405031 23533 raft_consensus.cc:2379] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4109ccbef0b043068afd24ef7bdec85e in current term 1: Already voted for candidate f30f957fba8444739c7c1f7b3fe41f02 in this term.
I20260620 18:44:57.405045 23751 leader_election.cc:288] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [CANDIDATE]: Term 1 election: Requested vote from peers 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975), bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647)
I20260620 18:44:57.405205 23393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c30730cf28a14d01b6bd18a6001431f6" candidate_uuid: "f30f957fba8444739c7c1f7b3fe41f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a"
I20260620 18:44:57.405223 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c30730cf28a14d01b6bd18a6001431f6" candidate_uuid: "f30f957fba8444739c7c1f7b3fe41f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109ccbef0b043068afd24ef7bdec85e"
I20260620 18:44:57.405275 23606 leader_election.cc:302] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4109ccbef0b043068afd24ef7bdec85e; no voters: bdfd69d052cf428ea35e6e4b4d7abb7a, f30f957fba8444739c7c1f7b3fe41f02
I20260620 18:44:57.405287 23393 raft_consensus.cc:3037] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:57.405297 23673 raft_consensus.cc:3037] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:57.406080 23393 raft_consensus.cc:2454] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30f957fba8444739c7c1f7b3fe41f02 in term 1.
I20260620 18:44:57.406227 23673 raft_consensus.cc:2454] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30f957fba8444739c7c1f7b3fe41f02 in term 1.
I20260620 18:44:57.406265 23464 leader_election.cc:302] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdfd69d052cf428ea35e6e4b4d7abb7a, f30f957fba8444739c7c1f7b3fe41f02; no voters:
I20260620 18:44:57.406296 23753 raft_consensus.cc:2726] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 18:44:57.406345 23751 raft_consensus.cc:2781] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:57.406508 23751 raft_consensus.cc:686] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 LEADER]: Becoming Leader. State: Replica: f30f957fba8444739c7c1f7b3fe41f02, State: Running, Role: LEADER
I20260620 18:44:57.406635 23751 consensus_queue.cc:237] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } }
I20260620 18:44:57.407518 23236 catalog_manager.cc:5495] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66). New cstate: current_term: 1 leader_uuid: "f30f957fba8444739c7c1f7b3fe41f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } health_report { overall_health: HEALTHY } } }
W20260620 18:44:57.413707 23720 tablet.cc:2369] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:44:57.448053 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.445617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53318 (local address 127.15.77.66:37863)
0620 18:44:57.445684 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.445685 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.445693 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:44:57.445731 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.445731 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.445750 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.445816 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:57.445942 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.446409 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.447187 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.447443 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.447543 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.447737 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:57.447857 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:57.447931 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.447969 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.447987 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 18:44:57.450042 23673 raft_consensus.cc:1261] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Refusing update from remote peer f30f957fba8444739c7c1f7b3fe41f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:57.450018 23393 raft_consensus.cc:1261] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Refusing update from remote peer f30f957fba8444739c7c1f7b3fe41f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:57.450395 23751 consensus_queue.cc:1035] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:57.450562 23759 consensus_queue.cc:1035] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:57.452831 23762 mvcc.cc:205] Tried to move back new op lower bound from 7298994575152254976 to 7298994574978183168. Current Snapshot: MvccSnapshot[applied={T|T < 7298994575152254976}]
I20260620 18:44:57.453081 23764 mvcc.cc:205] Tried to move back new op lower bound from 7298994575152254976 to 7298994574978183168. Current Snapshot: MvccSnapshot[applied={T|T < 7298994575152254976}]
I20260620 18:44:57.455727 23235 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:57.455823 23235 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:57.456593 23235 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407: Sending DeleteTablet for 3 replicas of tablet c30730cf28a14d01b6bd18a6001431f6
I20260620 18:44:57.456938 23373 tablet_service.cc:1514] Processing DeleteTablet for tablet c30730cf28a14d01b6bd18a6001431f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:57 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:43476
I20260620 18:44:57.456938 23513 tablet_service.cc:1514] Processing DeleteTablet for tablet c30730cf28a14d01b6bd18a6001431f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:57 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:53308
I20260620 18:44:57.457082 23235 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:44:57.457230 23235 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:57.457235 23634 tablet_service.cc:1514] Processing DeleteTablet for tablet c30730cf28a14d01b6bd18a6001431f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:57 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:46704
I20260620 18:44:57.457556 23773 tablet_replica.cc:330] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: stopping tablet replica
I20260620 18:44:57.457700 23773 raft_consensus.cc:2229] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:57.457813 23773 raft_consensus.cc:2258] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:57.458443 23373 tablet_service.cc:1467] Processing CreateTablet for tablet 0581cd53f1e24f37b171c0f261cd79d4 (DEFAULT_TABLE table=test-table [id=c90242d6244b44fab977f2b389fdd5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:57.458463 23513 tablet_service.cc:1467] Processing CreateTablet for tablet 0581cd53f1e24f37b171c0f261cd79d4 (DEFAULT_TABLE table=test-table [id=c90242d6244b44fab977f2b389fdd5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:57.458458 23773 ts_tablet_manager.cc:1905] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:57.458565 23513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0581cd53f1e24f37b171c0f261cd79d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:57.458565 23373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0581cd53f1e24f37b171c0f261cd79d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:57.459923 23747 tablet_bootstrap.cc:492] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02: Bootstrap starting.
I20260620 18:44:57.459934 23752 tablet_bootstrap.cc:492] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a: Bootstrap starting.
I20260620 18:44:57.460027 23634 tablet_service.cc:1467] Processing CreateTablet for tablet 0581cd53f1e24f37b171c0f261cd79d4 (DEFAULT_TABLE table=test-table [id=c90242d6244b44fab977f2b389fdd5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:57.460150 23634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0581cd53f1e24f37b171c0f261cd79d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:57.460234 23771 tablet_replica.cc:330] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: stopping tablet replica
I20260620 18:44:57.460325 23771 raft_consensus.cc:2229] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:44:57.460359 23747 tablet_bootstrap.cc:654] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:57.460359 23752 tablet_bootstrap.cc:654] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:57.460445 23771 raft_consensus.cc:2258] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:57.460582 23772 tablet_replica.cc:330] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: stopping tablet replica
I20260620 18:44:57.460680 23772 raft_consensus.cc:2229] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:57.460768 23772 raft_consensus.cc:2258] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:57.461267 23752 tablet_bootstrap.cc:492] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a: No bootstrap required, opened a new log
I20260620 18:44:57.461313 23752 ts_tablet_manager.cc:1397] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:57.461462 23749 tablet_bootstrap.cc:492] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e: Bootstrap starting.
I20260620 18:44:57.461457 23752 raft_consensus.cc:348] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.461540 23752 raft_consensus.cc:374] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:57.461561 23752 raft_consensus.cc:729] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfd69d052cf428ea35e6e4b4d7abb7a, State: Initialized, Role: FOLLOWER
I20260620 18:44:57.461602 23752 consensus_queue.cc:260] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.461688 23752 ts_tablet_manager.cc:1428] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:57.461863 23749 tablet_bootstrap.cc:654] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:57.460158 23773 ts_tablet_manager.cc:1918] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:57.462162 23773 log.cc:1196] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/c30730cf28a14d01b6bd18a6001431f6
I20260620 18:44:57.462419 23773 ts_tablet_manager.cc:1939] T c30730cf28a14d01b6bd18a6001431f6 P 4109ccbef0b043068afd24ef7bdec85e: Deleting consensus metadata
I20260620 18:44:57.462404 23772 ts_tablet_manager.cc:1905] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:57.462508 23771 ts_tablet_manager.cc:1905] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:57.462853 23222 catalog_manager.cc:4841] TS 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975): tablet c30730cf28a14d01b6bd18a6001431f6 (table test-table [id=bd5e0bb9956c4aba9c509994e1b51b3e]) successfully deleted
I20260620 18:44:57.463109 23747 tablet_bootstrap.cc:492] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02: No bootstrap required, opened a new log
I20260620 18:44:57.463181 23747 ts_tablet_manager.cc:1397] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260620 18:44:57.463354 23747 raft_consensus.cc:348] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.463446 23747 raft_consensus.cc:374] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:57.463476 23747 raft_consensus.cc:729] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30f957fba8444739c7c1f7b3fe41f02, State: Initialized, Role: FOLLOWER
I20260620 18:44:57.463521 23747 consensus_queue.cc:260] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.463606 23747 ts_tablet_manager.cc:1428] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:57.464000 23771 ts_tablet_manager.cc:1918] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:57.464015 23772 ts_tablet_manager.cc:1918] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:57.464077 23771 log.cc:1196] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/c30730cf28a14d01b6bd18a6001431f6
I20260620 18:44:57.464077 23772 log.cc:1196] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/c30730cf28a14d01b6bd18a6001431f6
I20260620 18:44:57.464341 23772 ts_tablet_manager.cc:1939] T c30730cf28a14d01b6bd18a6001431f6 P bdfd69d052cf428ea35e6e4b4d7abb7a: Deleting consensus metadata
I20260620 18:44:57.464341 23771 ts_tablet_manager.cc:1939] T c30730cf28a14d01b6bd18a6001431f6 P f30f957fba8444739c7c1f7b3fe41f02: Deleting consensus metadata
I20260620 18:44:57.464721 23222 catalog_manager.cc:4841] TS f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863): tablet c30730cf28a14d01b6bd18a6001431f6 (table test-table [id=bd5e0bb9956c4aba9c509994e1b51b3e]) successfully deleted
I20260620 18:44:57.464781 23220 catalog_manager.cc:4841] TS bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647): tablet c30730cf28a14d01b6bd18a6001431f6 (table test-table [id=bd5e0bb9956c4aba9c509994e1b51b3e]) successfully deleted
I20260620 18:44:57.465013 23749 tablet_bootstrap.cc:492] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e: No bootstrap required, opened a new log
I20260620 18:44:57.465075 23749 ts_tablet_manager.cc:1397] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 18:44:57.465225 23749 raft_consensus.cc:348] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.465299 23749 raft_consensus.cc:374] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:57.465332 23749 raft_consensus.cc:729] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4109ccbef0b043068afd24ef7bdec85e, State: Initialized, Role: FOLLOWER
I20260620 18:44:57.465382 23749 consensus_queue.cc:260] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.465468 23749 ts_tablet_manager.cc:1428] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:57.726147 23753 raft_consensus.cc:482] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:57.726260 23753 raft_consensus.cc:504] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.726503 23753 leader_election.cc:288] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647), f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863)
I20260620 18:44:57.726711 23393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0581cd53f1e24f37b171c0f261cd79d4" candidate_uuid: "4109ccbef0b043068afd24ef7bdec85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" is_pre_election: true
I20260620 18:44:57.726835 23393 raft_consensus.cc:2454] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4109ccbef0b043068afd24ef7bdec85e in term 0.
I20260620 18:44:57.726790 23533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0581cd53f1e24f37b171c0f261cd79d4" candidate_uuid: "4109ccbef0b043068afd24ef7bdec85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30f957fba8444739c7c1f7b3fe41f02" is_pre_election: true
I20260620 18:44:57.726928 23533 raft_consensus.cc:2454] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4109ccbef0b043068afd24ef7bdec85e in term 0.
I20260620 18:44:57.727047 23604 leader_election.cc:302] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4109ccbef0b043068afd24ef7bdec85e, bdfd69d052cf428ea35e6e4b4d7abb7a; no voters:
I20260620 18:44:57.727141 23753 raft_consensus.cc:2781] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:57.727191 23753 raft_consensus.cc:482] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:57.727207 23753 raft_consensus.cc:3037] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:57.727890 23753 raft_consensus.cc:504] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.728029 23753 leader_election.cc:288] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [CANDIDATE]: Term 1 election: Requested vote from peers bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647), f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863)
I20260620 18:44:57.728206 23393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0581cd53f1e24f37b171c0f261cd79d4" candidate_uuid: "4109ccbef0b043068afd24ef7bdec85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a"
I20260620 18:44:57.728240 23533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0581cd53f1e24f37b171c0f261cd79d4" candidate_uuid: "4109ccbef0b043068afd24ef7bdec85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30f957fba8444739c7c1f7b3fe41f02"
I20260620 18:44:57.728302 23393 raft_consensus.cc:3037] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:57.728317 23533 raft_consensus.cc:3037] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:57.728928 23393 raft_consensus.cc:2454] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4109ccbef0b043068afd24ef7bdec85e in term 1.
I20260620 18:44:57.729139 23604 leader_election.cc:302] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4109ccbef0b043068afd24ef7bdec85e, bdfd69d052cf428ea35e6e4b4d7abb7a; no voters:
I20260620 18:44:57.729197 23533 raft_consensus.cc:2454] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4109ccbef0b043068afd24ef7bdec85e in term 1.
I20260620 18:44:57.729259 23753 raft_consensus.cc:2781] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:57.729416 23753 raft_consensus.cc:686] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 1 LEADER]: Becoming Leader. State: Replica: 4109ccbef0b043068afd24ef7bdec85e, State: Running, Role: LEADER
I20260620 18:44:57.729496 23753 consensus_queue.cc:237] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.730171 23236 catalog_manager.cc:5495] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67). New cstate: current_term: 1 leader_uuid: "4109ccbef0b043068afd24ef7bdec85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } health_report { overall_health: HEALTHY } } }
Jun 20 18:44:57 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981096, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.126@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:57.774755 23784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.768090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41979 (local address 127.0.0.1:60960)
0620 18:44:57.768282 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:57.768298 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:57.768371 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:57.768562 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:57.768563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:57.768863 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:44:57.768919 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.768921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.769570 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.769571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.770111 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:57.770113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.770438 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.770439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:57.770482 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.771003 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:44:57.771007 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:44:57.772392 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:44:57.773817 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:57.773824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:57.773826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:57.774025 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:57.774207 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:44:57.774208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:44:57.774208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:44:57.774264 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:44:57.774468 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:44:57.774474 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:44:57.774515 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 18:44:57.774565 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 18:44:57.774656 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260620 18:44:57.774833 23785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:57.768151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.15.77.126:41979)
0620 18:44:57.768345 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:57.768347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:57.768357 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:44:57.768429 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:57.768430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:57.768440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:57.768553 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:57.768992 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.769493 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.770189 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:57.770371 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:57.770422 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:57.772472 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:57.772473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:57.772473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:57.772479 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:57.773744 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:57.774087 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:57.774088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:57.774088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:57.774141 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:57.774314 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:57.774315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:57.774315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:57.774416 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:57.774465 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:57.774602 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 18:44:57.774703 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":169,"thread_start_us":135,"threads_started":1}
I20260620 18:44:57.776757 23236 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:60960:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 18:44:57.776947 23236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:57.778378 23373 tablet_service.cc:1467] Processing CreateTablet for tablet 0eb2866c4d924c168143fd7f65b20cf1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55aff7abb65642f6b0183f152e819080]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 18:44:57.778415 23513 tablet_service.cc:1467] Processing CreateTablet for tablet 0eb2866c4d924c168143fd7f65b20cf1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55aff7abb65642f6b0183f152e819080]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 18:44:57.778388 23634 tablet_service.cc:1467] Processing CreateTablet for tablet 0eb2866c4d924c168143fd7f65b20cf1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55aff7abb65642f6b0183f152e819080]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 18:44:57.778571 23634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eb2866c4d924c168143fd7f65b20cf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:57.778574 23513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eb2866c4d924c168143fd7f65b20cf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:57.778573 23373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eb2866c4d924c168143fd7f65b20cf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:57.779896 23752 tablet_bootstrap.cc:492] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: Bootstrap starting.
I20260620 18:44:57.780128 23747 tablet_bootstrap.cc:492] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: Bootstrap starting.
I20260620 18:44:57.780352 23752 tablet_bootstrap.cc:654] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:57.780478 23747 tablet_bootstrap.cc:654] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:57.780762 23749 tablet_bootstrap.cc:492] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: Bootstrap starting.
I20260620 18:44:57.781186 23749 tablet_bootstrap.cc:654] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:57.781198 23747 tablet_bootstrap.cc:492] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: No bootstrap required, opened a new log
I20260620 18:44:57.781246 23747 ts_tablet_manager.cc:1397] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:57.781376 23747 raft_consensus.cc:348] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.781445 23747 raft_consensus.cc:374] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:57.781459 23747 raft_consensus.cc:729] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30f957fba8444739c7c1f7b3fe41f02, State: Initialized, Role: FOLLOWER
I20260620 18:44:57.781512 23747 consensus_queue.cc:260] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.781620 23747 ts_tablet_manager.cc:1428] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:57.781641 23759 tablet_replica.cc:439] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } }
I20260620 18:44:57.781739 23759 tablet_replica.cc:442] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:57.781831 23752 tablet_bootstrap.cc:492] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: No bootstrap required, opened a new log
I20260620 18:44:57.781869 23752 ts_tablet_manager.cc:1397] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:44:57.782006 23752 raft_consensus.cc:348] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.782094 23752 raft_consensus.cc:374] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:57.782119 23752 raft_consensus.cc:729] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfd69d052cf428ea35e6e4b4d7abb7a, State: Initialized, Role: FOLLOWER
I20260620 18:44:57.782181 23749 tablet_bootstrap.cc:492] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: No bootstrap required, opened a new log
I20260620 18:44:57.782167 23752 consensus_queue.cc:260] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.782226 23749 ts_tablet_manager.cc:1397] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:57.782260 23752 ts_tablet_manager.cc:1428] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:57.782364 23749 raft_consensus.cc:348] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.782440 23749 raft_consensus.cc:374] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:57.782401 23755 tablet_replica.cc:439] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } }
I20260620 18:44:57.782469 23749 raft_consensus.cc:729] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4109ccbef0b043068afd24ef7bdec85e, State: Initialized, Role: FOLLOWER
I20260620 18:44:57.782474 23755 tablet_replica.cc:442] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:57.782526 23749 consensus_queue.cc:260] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:57.782627 23749 ts_tablet_manager.cc:1428] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:57.782672 23753 tablet_replica.cc:439] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } }
I20260620 18:44:57.782783 23753 tablet_replica.cc:442] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:58.002220 23755 raft_consensus.cc:482] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:58.002337 23755 raft_consensus.cc:504] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.002781 23755 leader_election.cc:288] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863), 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975)
I20260620 18:44:58.006117 23789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.002993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:37863 (local address 127.15.77.65:53231)
0620 18:44:58.003179 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:58.003194 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:58.003274 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:58.003487 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:58.003488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:58.003491 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:58.003559 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.003561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.004251 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.004252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.005133 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.005136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.005811 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.005812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.005884 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.005890 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 18:44:58.005946 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 18:44:58.006002 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":58,"thread_start_us":108,"threads_started":1}
I20260620 18:44:58.006147 23792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.003338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:41211 (local address 127.15.77.67:39975)
0620 18:44:58.003436 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.003438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.003463 (+ 25us) server_negotiation.cc:407] Connection header received
0620 18:44:58.003491 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.003492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.003503 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.003576 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:58.003736 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.004196 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.005240 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.005671 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.005801 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.005811 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.006026 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.006074 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":56,"threads_started":1}
I20260620 18:44:58.006188 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.003072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:53231 (local address 127.15.77.66:37863)
0620 18:44:58.003233 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.003236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.003257 (+ 21us) server_negotiation.cc:407] Connection header received
0620 18:44:58.003375 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.003377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.003389 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.003467 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:44:58.003638 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.004170 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.005239 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.005647 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.005743 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.005757 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.006052 (+ 295us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.006104 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260620 18:44:58.006338 23791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.003241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:39975 (local address 127.15.77.65:41211)
0620 18:44:58.003346 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:58.003357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:58.003414 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:58.003589 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:58.003590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:58.003609 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:44:58.003662 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.003663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.004311 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.004312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.005137 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.005139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.005812 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.005812 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.005884 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.005888 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:44:58.005948 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 18:44:58.006007 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":6,"thread_start_us":44,"threads_started":1}
I20260620 18:44:58.006531 23533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb2866c4d924c168143fd7f65b20cf1" candidate_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30f957fba8444739c7c1f7b3fe41f02" is_pre_election: true
I20260620 18:44:58.006633 23533 raft_consensus.cc:2454] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdfd69d052cf428ea35e6e4b4d7abb7a in term 0.
I20260620 18:44:58.006812 23326 leader_election.cc:302] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdfd69d052cf428ea35e6e4b4d7abb7a, f30f957fba8444739c7c1f7b3fe41f02; no voters:
I20260620 18:44:58.006819 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb2866c4d924c168143fd7f65b20cf1" candidate_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109ccbef0b043068afd24ef7bdec85e" is_pre_election: true
I20260620 18:44:58.006924 23673 raft_consensus.cc:2454] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdfd69d052cf428ea35e6e4b4d7abb7a in term 0.
I20260620 18:44:58.006965 23755 raft_consensus.cc:2781] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:58.007015 23755 raft_consensus.cc:482] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:58.007048 23755 raft_consensus.cc:3037] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:58.007827 23755 raft_consensus.cc:504] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.007985 23755 leader_election.cc:288] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [CANDIDATE]: Term 1 election: Requested vote from peers f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863), 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975)
I20260620 18:44:58.008138 23533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb2866c4d924c168143fd7f65b20cf1" candidate_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30f957fba8444739c7c1f7b3fe41f02"
I20260620 18:44:58.008178 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eb2866c4d924c168143fd7f65b20cf1" candidate_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109ccbef0b043068afd24ef7bdec85e"
I20260620 18:44:58.008244 23533 raft_consensus.cc:3037] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:58.008239 23673 raft_consensus.cc:3037] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:58.008836 23673 raft_consensus.cc:2454] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdfd69d052cf428ea35e6e4b4d7abb7a in term 1.
I20260620 18:44:58.008942 23533 raft_consensus.cc:2454] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdfd69d052cf428ea35e6e4b4d7abb7a in term 1.
I20260620 18:44:58.009006 23326 leader_election.cc:302] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4109ccbef0b043068afd24ef7bdec85e, bdfd69d052cf428ea35e6e4b4d7abb7a; no voters:
I20260620 18:44:58.009126 23755 raft_consensus.cc:2781] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:58.009268 23755 raft_consensus.cc:686] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 LEADER]: Becoming Leader. State: Replica: bdfd69d052cf428ea35e6e4b4d7abb7a, State: Running, Role: LEADER
I20260620 18:44:58.009338 23755 consensus_queue.cc:237] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.009670 23793 tablet_replica.cc:439] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: TxnStatusTablet state changed. Reason: New leader bdfd69d052cf428ea35e6e4b4d7abb7a. Latest consensus state: current_term: 1 leader_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } }
I20260620 18:44:58.009743 23793 tablet_replica.cc:442] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: This TxnStatusTablet replica's current role is: LEADER
I20260620 18:44:58.009967 23795 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 18:44:58.010095 23795 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 18:44:58.010125 23236 catalog_manager.cc:5495] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a reported cstate change: term changed from 0 to 1, leader changed from <none> to bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65). New cstate: current_term: 1 leader_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } health_report { overall_health: UNKNOWN } } }
I20260620 18:44:58.067356 23798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.064688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.15.77.65:33647)
0620 18:44:58.064849 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.064851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.064857 (+ 6us) server_negotiation.cc:407] Connection header received
0620 18:44:58.064903 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.064903 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.064920 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.065004 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:58.065147 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.065726 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.066487 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.066705 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.066792 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.066985 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.067086 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.067162 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.067220 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.067242 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260620 18:44:58.067355 23784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.064600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33647 (local address 127.0.0.1:43488)
0620 18:44:58.064687 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:58.064702 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:58.064758 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:58.065009 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:58.065010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:58.065011 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:44:58.065073 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.065075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.065812 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.065813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.066395 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.066397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.066796 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.066797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.066859 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.066875 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.067159 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.067168 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 18:44:58.067205 (+ 37us) client_negotiation.cc:240] Negotiation successful
0620 18:44:58.067264 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 18:44:58.068748 23673 raft_consensus.cc:1261] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Refusing update from remote peer bdfd69d052cf428ea35e6e4b4d7abb7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:58.068876 23533 raft_consensus.cc:1261] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Refusing update from remote peer bdfd69d052cf428ea35e6e4b4d7abb7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:58.069226 23755 consensus_queue.cc:1035] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:58.069378 23793 consensus_queue.cc:1035] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:58.069994 23753 tablet_replica.cc:439] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: TxnStatusTablet state changed. Reason: New leader bdfd69d052cf428ea35e6e4b4d7abb7a. Latest consensus state: current_term: 1 leader_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } }
I20260620 18:44:58.070111 23753 tablet_replica.cc:442] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:58.070230 23759 tablet_replica.cc:439] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: TxnStatusTablet state changed. Reason: New leader bdfd69d052cf428ea35e6e4b4d7abb7a. Latest consensus state: current_term: 1 leader_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } }
I20260620 18:44:58.070327 23759 tablet_replica.cc:442] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:58.071219 23793 tablet_replica.cc:439] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } }
I20260620 18:44:58.071369 23759 tablet_replica.cc:439] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } }
I20260620 18:44:58.071449 23759 tablet_replica.cc:442] T 0eb2866c4d924c168143fd7f65b20cf1 P f30f957fba8444739c7c1f7b3fe41f02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:58.071416 23753 tablet_replica.cc:439] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } }
I20260620 18:44:58.071486 23753 tablet_replica.cc:442] T 0eb2866c4d924c168143fd7f65b20cf1 P 4109ccbef0b043068afd24ef7bdec85e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 18:44:58.072119 23800 mvcc.cc:205] Tried to move back new op lower bound from 7298994577687465984 to 7298994577446846464. Current Snapshot: MvccSnapshot[applied={T|T < 7298994577687465984}]
I20260620 18:44:58.071300 23793 tablet_replica.cc:442] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: This TxnStatusTablet replica's current role is: LEADER
I20260620 18:44:58.071877 23755 tablet_replica.cc:439] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } } }
I20260620 18:44:58.073165 23755 tablet_replica.cc:442] T 0eb2866c4d924c168143fd7f65b20cf1 P bdfd69d052cf428ea35e6e4b4d7abb7a: This TxnStatusTablet replica's current role is: LEADER
I20260620 18:44:58.075861 23792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.073424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.15.77.67:39975)
0620 18:44:58.073465 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.073468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.073513 (+ 45us) server_negotiation.cc:407] Connection header received
0620 18:44:58.073597 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.073598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.073618 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.073713 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:58.073829 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.074287 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.075051 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.075260 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.075351 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.075526 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.075628 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.075715 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.075759 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.075781 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:44:58.081944 23813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.079215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33647 (local address 127.15.77.67:46121)
0620 18:44:58.079381 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:58.079393 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:58.079464 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:58.079595 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:58.079595 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:58.079597 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:44:58.079638 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.079639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.080507 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.080508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.081074 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.081076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.081556 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.081557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.081603 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.081616 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.081807 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.081809 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:44:58.081846 (+ 37us) client_negotiation.cc:240] Negotiation successful
0620 18:44:58.081885 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260620 18:44:58.081952 23798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.079307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:46121 (local address 127.15.77.65:33647)
0620 18:44:58.079386 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.079388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.079475 (+ 87us) server_negotiation.cc:407] Connection header received
0620 18:44:58.079506 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.079507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.079519 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.079587 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:58.079705 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.080414 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.081165 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.081473 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.081553 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.081698 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.081735 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.081793 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.081850 (+ 57us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.081874 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 18:44:58.084882 23393 raft_consensus.cc:1261] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Refusing update from remote peer 4109ccbef0b043068afd24ef7bdec85e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:58.085052 23533 raft_consensus.cc:1261] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Refusing update from remote peer 4109ccbef0b043068afd24ef7bdec85e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:58.085160 23777 consensus_queue.cc:1035] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:58.085318 23753 consensus_queue.cc:1035] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:58.098322 23792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.091688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:37073 (local address 127.15.77.67:39975)
0620 18:44:58.091732 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.091735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.095588 (+ 3853us) server_negotiation.cc:407] Connection header received
0620 18:44:58.095635 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.095636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.095648 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.095715 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:58.095887 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.096449 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.097636 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.097822 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.097870 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.098091 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.098117 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.098155 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.098240 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.098259 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:44:58.098397 23817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.091609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:39975 (local address 127.15.77.65:37073)
0620 18:44:58.095488 (+ 3879us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:58.095504 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:58.095583 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:58.095724 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:58.095724 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:58.095726 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:44:58.095812 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.095814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.096540 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.096541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.097538 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.097541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.097900 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.097901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.097980 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.098002 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.098179 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.098181 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:44:58.098238 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 18:44:58.098288 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3818,"thread_start_us":91,"threads_started":1}
I20260620 18:44:58.108741 23236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:58.108875 23236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:58.109598 23236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407: Sending DeleteTablet for 3 replicas of tablet 0581cd53f1e24f37b171c0f261cd79d4
I20260620 18:44:58.109935 23634 tablet_service.cc:1514] Processing DeleteTablet for tablet 0581cd53f1e24f37b171c0f261cd79d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:58 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:46704
I20260620 18:44:58.109999 23373 tablet_service.cc:1514] Processing DeleteTablet for tablet 0581cd53f1e24f37b171c0f261cd79d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:58 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:43476
I20260620 18:44:58.110196 23513 tablet_service.cc:1514] Processing DeleteTablet for tablet 0581cd53f1e24f37b171c0f261cd79d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:58 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:53308
I20260620 18:44:58.115443 23821 tablet_replica.cc:330] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e: stopping tablet replica
I20260620 18:44:58.115550 23821 raft_consensus.cc:2229] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:44:58.115648 23821 raft_consensus.cc:2258] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:58.115931 23821 ts_tablet_manager.cc:1905] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:58.117758 23822 tablet_replica.cc:330] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a: stopping tablet replica
I20260620 18:44:58.117843 23822 raft_consensus.cc:2229] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:58.117894 23822 raft_consensus.cc:2258] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:58.118060 23821 ts_tablet_manager.cc:1918] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:58.118122 23821 log.cc:1196] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/0581cd53f1e24f37b171c0f261cd79d4
I20260620 18:44:58.118343 23822 ts_tablet_manager.cc:1905] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:58.118366 23821 ts_tablet_manager.cc:1939] T 0581cd53f1e24f37b171c0f261cd79d4 P 4109ccbef0b043068afd24ef7bdec85e: Deleting consensus metadata
I20260620 18:44:58.118705 23823 tablet_replica.cc:330] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02: stopping tablet replica
I20260620 18:44:58.118775 23823 raft_consensus.cc:2229] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:58.118769 23222 catalog_manager.cc:4841] TS 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975): tablet 0581cd53f1e24f37b171c0f261cd79d4 (table test-table [id=c90242d6244b44fab977f2b389fdd5bb]) successfully deleted
I20260620 18:44:58.118816 23823 raft_consensus.cc:2258] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:58.119091 23823 ts_tablet_manager.cc:1905] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:58.120682 23822 ts_tablet_manager.cc:1918] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:58.120800 23822 log.cc:1196] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/0581cd53f1e24f37b171c0f261cd79d4
I20260620 18:44:58.121119 23823 ts_tablet_manager.cc:1918] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:58.121184 23823 log.cc:1196] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/0581cd53f1e24f37b171c0f261cd79d4
I20260620 18:44:58.121138 23822 ts_tablet_manager.cc:1939] T 0581cd53f1e24f37b171c0f261cd79d4 P bdfd69d052cf428ea35e6e4b4d7abb7a: Deleting consensus metadata
I20260620 18:44:58.121426 23823 ts_tablet_manager.cc:1939] T 0581cd53f1e24f37b171c0f261cd79d4 P f30f957fba8444739c7c1f7b3fe41f02: Deleting consensus metadata
I20260620 18:44:58.121713 23220 catalog_manager.cc:4841] TS bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647): tablet 0581cd53f1e24f37b171c0f261cd79d4 (table test-table [id=c90242d6244b44fab977f2b389fdd5bb]) successfully deleted
I20260620 18:44:58.121862 23222 catalog_manager.cc:4841] TS f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863): tablet 0581cd53f1e24f37b171c0f261cd79d4 (table test-table [id=c90242d6244b44fab977f2b389fdd5bb]) successfully deleted
Jun 20 18:44:58 dist-test-slave-tvl9 krb5kdc[23191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 18:44:58 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:44:58 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 18:44:58.148746 23785 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:44:58.141592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60972 (local address 127.15.77.126:41979)
0620 18:44:58.141643 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.141646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.141827 (+ 181us) server_negotiation.cc:407] Connection header received
0620 18:44:58.141876 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.141877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.141893 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.141965 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:58.142705 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.143561 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.144677 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.144986 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.145058 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.148674 (+ 3616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60972: BlockingRecv error: recv got EOF from 127.0.0.1:60972 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Jun 20 18:44:58 dist-test-slave-tvl9 krb5kdc[23191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM
I20260620 18:44:58.159020 23785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.149501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.15.77.126:41979)
0620 18:44:58.149549 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.149551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.149851 (+ 300us) server_negotiation.cc:407] Connection header received
0620 18:44:58.149914 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.149915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.149930 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.149999 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:44:58.150558 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.151596 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.152762 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.153061 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.153101 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.156070 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:44:58.156071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:44:58.156072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:44:58.156080 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:44:58.157791 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:58.158186 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:58.158187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:58.158188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:58.158240 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:44:58.158434 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:44:58.158434 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:44:58.158435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:44:58.158558 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:44:58.158630 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.158830 (+ 200us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.158915 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:44:58.161051 23236 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:44:58.161223 23236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:58.162878 23634 tablet_service.cc:1467] Processing CreateTablet for tablet f954c5e99d1e400e9a80633bbfba46dd (DEFAULT_TABLE table=test-table [id=4c7ea0164e66406cad77cba0a94b3e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:58.163009 23634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f954c5e99d1e400e9a80633bbfba46dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:58.163100 23513 tablet_service.cc:1467] Processing CreateTablet for tablet f954c5e99d1e400e9a80633bbfba46dd (DEFAULT_TABLE table=test-table [id=4c7ea0164e66406cad77cba0a94b3e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:58.163229 23513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f954c5e99d1e400e9a80633bbfba46dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:58.164517 23749 tablet_bootstrap.cc:492] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e: Bootstrap starting.
I20260620 18:44:58.164682 23747 tablet_bootstrap.cc:492] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02: Bootstrap starting.
I20260620 18:44:58.164830 23749 tablet_bootstrap.cc:654] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:58.165009 23747 tablet_bootstrap.cc:654] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:58.165565 23373 tablet_service.cc:1467] Processing CreateTablet for tablet f954c5e99d1e400e9a80633bbfba46dd (DEFAULT_TABLE table=test-table [id=4c7ea0164e66406cad77cba0a94b3e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:58.165692 23373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f954c5e99d1e400e9a80633bbfba46dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:58.165956 23749 tablet_bootstrap.cc:492] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e: No bootstrap required, opened a new log
I20260620 18:44:58.166002 23749 ts_tablet_manager.cc:1397] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:44:58.166138 23749 raft_consensus.cc:348] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.166213 23749 raft_consensus.cc:374] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:58.166234 23749 raft_consensus.cc:729] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4109ccbef0b043068afd24ef7bdec85e, State: Initialized, Role: FOLLOWER
I20260620 18:44:58.166272 23749 consensus_queue.cc:260] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.166356 23749 ts_tablet_manager.cc:1428] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:58.166947 23752 tablet_bootstrap.cc:492] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a: Bootstrap starting.
I20260620 18:44:58.167301 23752 tablet_bootstrap.cc:654] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:58.167532 23747 tablet_bootstrap.cc:492] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02: No bootstrap required, opened a new log
I20260620 18:44:58.167588 23747 ts_tablet_manager.cc:1397] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 18:44:58.167703 23747 raft_consensus.cc:348] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.167774 23747 raft_consensus.cc:374] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:58.167794 23747 raft_consensus.cc:729] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30f957fba8444739c7c1f7b3fe41f02, State: Initialized, Role: FOLLOWER
I20260620 18:44:58.167840 23747 consensus_queue.cc:260] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.167917 23752 tablet_bootstrap.cc:492] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a: No bootstrap required, opened a new log
I20260620 18:44:58.167923 23747 ts_tablet_manager.cc:1428] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:58.167960 23752 ts_tablet_manager.cc:1397] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 18:44:58.168054 23752 raft_consensus.cc:348] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.168116 23752 raft_consensus.cc:374] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:58.168138 23752 raft_consensus.cc:729] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfd69d052cf428ea35e6e4b4d7abb7a, State: Initialized, Role: FOLLOWER
I20260620 18:44:58.168183 23752 consensus_queue.cc:260] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.168267 23752 ts_tablet_manager.cc:1428] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:58.301519 23808 raft_consensus.cc:482] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:58.301635 23808 raft_consensus.cc:504] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.301834 23808 leader_election.cc:288] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863), 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975)
I20260620 18:44:58.302124 23533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f954c5e99d1e400e9a80633bbfba46dd" candidate_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30f957fba8444739c7c1f7b3fe41f02" is_pre_election: true
I20260620 18:44:58.302253 23533 raft_consensus.cc:2454] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdfd69d052cf428ea35e6e4b4d7abb7a in term 0.
I20260620 18:44:58.302237 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f954c5e99d1e400e9a80633bbfba46dd" candidate_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109ccbef0b043068afd24ef7bdec85e" is_pre_election: true
I20260620 18:44:58.302389 23673 raft_consensus.cc:2454] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdfd69d052cf428ea35e6e4b4d7abb7a in term 0.
I20260620 18:44:58.302465 23326 leader_election.cc:302] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdfd69d052cf428ea35e6e4b4d7abb7a, f30f957fba8444739c7c1f7b3fe41f02; no voters:
I20260620 18:44:58.302587 23808 raft_consensus.cc:2781] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:58.302644 23808 raft_consensus.cc:482] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:58.302664 23808 raft_consensus.cc:3037] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:58.303552 23808 raft_consensus.cc:504] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.303670 23808 leader_election.cc:288] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [CANDIDATE]: Term 1 election: Requested vote from peers f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863), 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975)
I20260620 18:44:58.303892 23533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f954c5e99d1e400e9a80633bbfba46dd" candidate_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30f957fba8444739c7c1f7b3fe41f02"
I20260620 18:44:58.303934 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f954c5e99d1e400e9a80633bbfba46dd" candidate_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109ccbef0b043068afd24ef7bdec85e"
I20260620 18:44:58.304001 23533 raft_consensus.cc:3037] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:58.304040 23673 raft_consensus.cc:3037] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:58.304652 23533 raft_consensus.cc:2454] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdfd69d052cf428ea35e6e4b4d7abb7a in term 1.
I20260620 18:44:58.304692 23673 raft_consensus.cc:2454] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdfd69d052cf428ea35e6e4b4d7abb7a in term 1.
I20260620 18:44:58.304878 23326 leader_election.cc:302] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4109ccbef0b043068afd24ef7bdec85e, bdfd69d052cf428ea35e6e4b4d7abb7a; no voters:
I20260620 18:44:58.305011 23808 raft_consensus.cc:2781] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:58.305075 23808 raft_consensus.cc:686] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 LEADER]: Becoming Leader. State: Replica: bdfd69d052cf428ea35e6e4b4d7abb7a, State: Running, Role: LEADER
I20260620 18:44:58.305135 23808 consensus_queue.cc:237] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.305840 23236 catalog_manager.cc:5495] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a reported cstate change: term changed from 0 to 1, leader changed from <none> to bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65). New cstate: current_term: 1 leader_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } health_report { overall_health: UNKNOWN } } }
I20260620 18:44:58.315573 23798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.313323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43502 (local address 127.15.77.65:33647)
0620 18:44:58.313394 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.313396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.313404 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:44:58.313442 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.313443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.313453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.313491 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:58.313621 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.314089 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.314811 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.315011 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.315080 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.315239 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.315266 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.315305 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.315444 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.315483 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 18:44:58.316558 23533 raft_consensus.cc:1261] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Refusing update from remote peer bdfd69d052cf428ea35e6e4b4d7abb7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:58.316563 23673 raft_consensus.cc:1261] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Refusing update from remote peer bdfd69d052cf428ea35e6e4b4d7abb7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:58.316812 23808 consensus_queue.cc:1035] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:58.316916 23793 consensus_queue.cc:1035] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:58.320077 23234 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:58.320179 23234 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:58.320843 23234 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407: Sending DeleteTablet for 3 replicas of tablet f954c5e99d1e400e9a80633bbfba46dd
I20260620 18:44:58.321173 23513 tablet_service.cc:1514] Processing DeleteTablet for tablet f954c5e99d1e400e9a80633bbfba46dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:58 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:53308
I20260620 18:44:58.321183 23373 tablet_service.cc:1514] Processing DeleteTablet for tablet f954c5e99d1e400e9a80633bbfba46dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:58 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:43476
I20260620 18:44:58.321277 23634 tablet_service.cc:1514] Processing DeleteTablet for tablet f954c5e99d1e400e9a80633bbfba46dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:58 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:46704
I20260620 18:44:58.321367 23822 tablet_replica.cc:330] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a: stopping tablet replica
I20260620 18:44:58.321427 23822 raft_consensus.cc:2229] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:44:58.321372 23234 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 18:44:58.321520 23822 raft_consensus.cc:2258] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 18:44:58.321527 23234 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:44:58.321619 23823 tablet_replica.cc:330] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02: stopping tablet replica
I20260620 18:44:58.321694 23823 raft_consensus.cc:2229] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:58.321758 23823 raft_consensus.cc:2258] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:58.321828 23821 tablet_replica.cc:330] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e: stopping tablet replica
I20260620 18:44:58.321889 23821 raft_consensus.cc:2229] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:58.321964 23821 raft_consensus.cc:2258] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:58.322551 23823 ts_tablet_manager.cc:1905] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:58.322762 23634 tablet_service.cc:1467] Processing CreateTablet for tablet ae6002a4f2504a83a3b66a531962a165 (DEFAULT_TABLE table=test-table [id=af25838e6228406085f58e2f437fc1fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:58.322813 23513 tablet_service.cc:1467] Processing CreateTablet for tablet ae6002a4f2504a83a3b66a531962a165 (DEFAULT_TABLE table=test-table [id=af25838e6228406085f58e2f437fc1fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:58.322880 23634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6002a4f2504a83a3b66a531962a165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:58.322916 23513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6002a4f2504a83a3b66a531962a165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:58.323218 23373 tablet_service.cc:1467] Processing CreateTablet for tablet ae6002a4f2504a83a3b66a531962a165 (DEFAULT_TABLE table=test-table [id=af25838e6228406085f58e2f437fc1fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:44:58.323504 23373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae6002a4f2504a83a3b66a531962a165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:44:58.324275 23749 tablet_bootstrap.cc:492] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e: Bootstrap starting.
I20260620 18:44:58.324437 23821 ts_tablet_manager.cc:1905] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:58.324599 23749 tablet_bootstrap.cc:654] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:58.325042 23752 tablet_bootstrap.cc:492] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a: Bootstrap starting.
I20260620 18:44:58.325325 23752 tablet_bootstrap.cc:654] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:58.325752 23821 ts_tablet_manager.cc:1918] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:58.325815 23821 log.cc:1196] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/f954c5e99d1e400e9a80633bbfba46dd
I20260620 18:44:58.325814 23823 ts_tablet_manager.cc:1918] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:58.325865 23823 log.cc:1196] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/f954c5e99d1e400e9a80633bbfba46dd
I20260620 18:44:58.326057 23821 ts_tablet_manager.cc:1939] T f954c5e99d1e400e9a80633bbfba46dd P 4109ccbef0b043068afd24ef7bdec85e: Deleting consensus metadata
I20260620 18:44:58.326113 23823 ts_tablet_manager.cc:1939] T f954c5e99d1e400e9a80633bbfba46dd P f30f957fba8444739c7c1f7b3fe41f02: Deleting consensus metadata
I20260620 18:44:58.326431 23752 tablet_bootstrap.cc:492] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a: No bootstrap required, opened a new log
I20260620 18:44:58.326483 23752 ts_tablet_manager.cc:1397] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260620 18:44:58.326493 23222 catalog_manager.cc:4841] TS f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863): tablet f954c5e99d1e400e9a80633bbfba46dd (table test-table [id=4c7ea0164e66406cad77cba0a94b3e52]) successfully deleted
I20260620 18:44:58.326613 23752 raft_consensus.cc:348] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.326701 23752 raft_consensus.cc:374] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:58.326725 23752 raft_consensus.cc:729] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfd69d052cf428ea35e6e4b4d7abb7a, State: Initialized, Role: FOLLOWER
I20260620 18:44:58.326772 23752 consensus_queue.cc:260] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.326864 23752 ts_tablet_manager.cc:1428] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 18:44:58.327061 23747 tablet_bootstrap.cc:492] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02: Bootstrap starting.
I20260620 18:44:58.327373 23747 tablet_bootstrap.cc:654] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02: Neither blocks nor log segments found. Creating new log.
I20260620 18:44:58.327379 23222 catalog_manager.cc:4841] TS 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975): tablet f954c5e99d1e400e9a80633bbfba46dd (table test-table [id=4c7ea0164e66406cad77cba0a94b3e52]) successfully deleted
I20260620 18:44:58.327719 23822 ts_tablet_manager.cc:1905] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:58.327867 23749 tablet_bootstrap.cc:492] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e: No bootstrap required, opened a new log
I20260620 18:44:58.327941 23749 ts_tablet_manager.cc:1397] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 18:44:58.328064 23749 raft_consensus.cc:348] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.328145 23749 raft_consensus.cc:374] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:58.328171 23749 raft_consensus.cc:729] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4109ccbef0b043068afd24ef7bdec85e, State: Initialized, Role: FOLLOWER
I20260620 18:44:58.328212 23749 consensus_queue.cc:260] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.328306 23749 ts_tablet_manager.cc:1428] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 18:44:58.329001 23822 ts_tablet_manager.cc:1918] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:44:58.329061 23822 log.cc:1196] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/f954c5e99d1e400e9a80633bbfba46dd
I20260620 18:44:58.329262 23822 ts_tablet_manager.cc:1939] T f954c5e99d1e400e9a80633bbfba46dd P bdfd69d052cf428ea35e6e4b4d7abb7a: Deleting consensus metadata
I20260620 18:44:58.329599 23220 catalog_manager.cc:4841] TS bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647): tablet f954c5e99d1e400e9a80633bbfba46dd (table test-table [id=4c7ea0164e66406cad77cba0a94b3e52]) successfully deleted
I20260620 18:44:58.329648 23747 tablet_bootstrap.cc:492] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02: No bootstrap required, opened a new log
I20260620 18:44:58.329710 23747 ts_tablet_manager.cc:1397] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 18:44:58.329806 23747 raft_consensus.cc:348] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.329870 23747 raft_consensus.cc:374] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:44:58.329898 23747 raft_consensus.cc:729] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30f957fba8444739c7c1f7b3fe41f02, State: Initialized, Role: FOLLOWER
I20260620 18:44:58.329936 23747 consensus_queue.cc:260] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.330021 23747 ts_tablet_manager.cc:1428] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:44:58.340360 23759 raft_consensus.cc:482] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:44:58.340482 23759 raft_consensus.cc:504] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.340689 23759 leader_election.cc:288] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647), 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975)
I20260620 18:44:58.340857 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6002a4f2504a83a3b66a531962a165" candidate_uuid: "f30f957fba8444739c7c1f7b3fe41f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109ccbef0b043068afd24ef7bdec85e" is_pre_election: true
I20260620 18:44:58.340969 23673 raft_consensus.cc:2454] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30f957fba8444739c7c1f7b3fe41f02 in term 0.
I20260620 18:44:58.340938 23393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6002a4f2504a83a3b66a531962a165" candidate_uuid: "f30f957fba8444739c7c1f7b3fe41f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" is_pre_election: true
I20260620 18:44:58.341037 23393 raft_consensus.cc:2454] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30f957fba8444739c7c1f7b3fe41f02 in term 0.
I20260620 18:44:58.341102 23466 leader_election.cc:302] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4109ccbef0b043068afd24ef7bdec85e, f30f957fba8444739c7c1f7b3fe41f02; no voters:
I20260620 18:44:58.341190 23759 raft_consensus.cc:2781] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:44:58.341252 23759 raft_consensus.cc:482] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:44:58.341284 23759 raft_consensus.cc:3037] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:58.342056 23759 raft_consensus.cc:504] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.342224 23759 leader_election.cc:288] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [CANDIDATE]: Term 1 election: Requested vote from peers bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647), 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975)
I20260620 18:44:58.342363 23393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6002a4f2504a83a3b66a531962a165" candidate_uuid: "f30f957fba8444739c7c1f7b3fe41f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a"
I20260620 18:44:58.342443 23393 raft_consensus.cc:3037] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:58.342424 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae6002a4f2504a83a3b66a531962a165" candidate_uuid: "f30f957fba8444739c7c1f7b3fe41f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109ccbef0b043068afd24ef7bdec85e"
I20260620 18:44:58.342576 23673 raft_consensus.cc:3037] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:44:58.343220 23393 raft_consensus.cc:2454] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30f957fba8444739c7c1f7b3fe41f02 in term 1.
I20260620 18:44:58.343358 23673 raft_consensus.cc:2454] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30f957fba8444739c7c1f7b3fe41f02 in term 1.
I20260620 18:44:58.343482 23464 leader_election.cc:302] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdfd69d052cf428ea35e6e4b4d7abb7a, f30f957fba8444739c7c1f7b3fe41f02; no voters:
I20260620 18:44:58.343573 23759 raft_consensus.cc:2781] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:44:58.343701 23759 raft_consensus.cc:686] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 LEADER]: Becoming Leader. State: Replica: f30f957fba8444739c7c1f7b3fe41f02, State: Running, Role: LEADER
I20260620 18:44:58.343766 23759 consensus_queue.cc:237] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } }
I20260620 18:44:58.344334 23234 catalog_manager.cc:5495] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66). New cstate: current_term: 1 leader_uuid: "f30f957fba8444739c7c1f7b3fe41f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f30f957fba8444739c7c1f7b3fe41f02" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 37863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 } health_report { overall_health: UNKNOWN } } }
I20260620 18:44:58.352216 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.349610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.15.77.66:37863)
0620 18:44:58.349657 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.349659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.349677 (+ 18us) server_negotiation.cc:407] Connection header received
0620 18:44:58.349738 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.349739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.349752 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.349812 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:58.349901 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.350642 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.351362 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.351682 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.351753 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.351912 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.351948 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.352001 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.352112 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.352136 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:44:58.357615 23798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.354775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:54041 (local address 127.15.77.65:33647)
0620 18:44:58.354827 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.354829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.355007 (+ 178us) server_negotiation.cc:407] Connection header received
0620 18:44:58.355032 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.355033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.355044 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.355089 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:58.355222 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.355917 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.356996 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.357177 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.357232 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.357406 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.357431 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.357471 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.357544 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.357559 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 18:44:58.357683 23851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.354716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33647 (local address 127.15.77.66:54041)
0620 18:44:58.354888 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:58.354899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:58.354957 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:58.355113 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:58.355114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:58.355116 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:44:58.355160 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.355161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.355991 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.355993 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.356904 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.356906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.357252 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.357253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.357313 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.357330 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.357488 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.357490 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:44:58.357543 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 18:44:58.357591 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260620 18:44:58.359612 23673 raft_consensus.cc:1261] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Refusing update from remote peer f30f957fba8444739c7c1f7b3fe41f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:58.359653 23393 raft_consensus.cc:1261] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Refusing update from remote peer f30f957fba8444739c7c1f7b3fe41f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:44:58.359856 23847 consensus_queue.cc:1035] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4109ccbef0b043068afd24ef7bdec85e" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 39975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:58.360014 23759 consensus_queue.cc:1035] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdfd69d052cf428ea35e6e4b4d7abb7a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:44:58.366994 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.364474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:42129 (local address 127.15.77.66:37863)
0620 18:44:58.364517 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:44:58.364519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:44:58.364528 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:44:58.364598 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:44:58.364598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:44:58.364607 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:44:58.364660 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:44:58.364786 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.365522 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.366215 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.366510 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.366575 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.366762 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.366795 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.366835 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 18:44:58.366906 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 18:44:58.366936 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:44:58.367019 23817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.364434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:37863 (local address 127.15.77.65:42129)
0620 18:44:58.364488 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:58.364499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:44:58.364536 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:44:58.364678 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:44:58.364679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:44:58.364680 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 18:44:58.364719 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.364721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.365593 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.365593 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.366138 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:44:58.366139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:44:58.366611 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:44:58.366612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:44:58.366682 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:44:58.366689 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 18:44:58.366854 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 18:44:58.366856 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:44:58.366901 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 18:44:58.366946 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 18:44:58.375679 23234 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:58.375782 23234 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:44:58.376427 23234 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 794c62ba67e440d699607c9814312407: Sending DeleteTablet for 3 replicas of tablet ae6002a4f2504a83a3b66a531962a165
I20260620 18:44:58.376643 23373 tablet_service.cc:1514] Processing DeleteTablet for tablet ae6002a4f2504a83a3b66a531962a165 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:58 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:43476
I20260620 18:44:58.376739 23822 tablet_replica.cc:330] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a: stopping tablet replica
I20260620 18:44:58.376744 23513 tablet_service.cc:1514] Processing DeleteTablet for tablet ae6002a4f2504a83a3b66a531962a165 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:58 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:53308
I20260620 18:44:58.376789 23822 raft_consensus.cc:2229] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:58.376816 23823 tablet_replica.cc:330] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02: stopping tablet replica
I20260620 18:44:58.376825 23822 raft_consensus.cc:2258] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:58.376873 23823 raft_consensus.cc:2229] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:44:58.376924 23823 raft_consensus.cc:2258] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:58.377136 23634 tablet_service.cc:1514] Processing DeleteTablet for tablet ae6002a4f2504a83a3b66a531962a165 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:44:58 UTC) from {username='oryx', principal='oryx/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:46704
I20260620 18:44:58.377228 23821 tablet_replica.cc:330] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e: stopping tablet replica
I20260620 18:44:58.377251 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 23308
I20260620 18:44:58.377262 23822 ts_tablet_manager.cc:1905] T ae6002a4f2504a83a3b66a531962a165 P bdfd69d052cf428ea35e6e4b4d7abb7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:58.377278 23821 raft_consensus.cc:2229] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:44:58.377310 23821 raft_consensus.cc:2258] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:44:58.378091 23821 ts_tablet_manager.cc:1905] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:58.378091 23823 ts_tablet_manager.cc:1905] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:44:58.379678 23823 ts_tablet_manager.cc:1918] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:58.379678 23821 ts_tablet_manager.cc:1918] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 18:44:58.379742 23823 log.cc:1196] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/ae6002a4f2504a83a3b66a531962a165
I20260620 18:44:58.379742 23821 log.cc:1196] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/ae6002a4f2504a83a3b66a531962a165
I20260620 18:44:58.379963 23821 ts_tablet_manager.cc:1939] T ae6002a4f2504a83a3b66a531962a165 P 4109ccbef0b043068afd24ef7bdec85e: Deleting consensus metadata
I20260620 18:44:58.379976 23823 ts_tablet_manager.cc:1939] T ae6002a4f2504a83a3b66a531962a165 P f30f957fba8444739c7c1f7b3fe41f02: Deleting consensus metadata
I20260620 18:44:58.380263 23222 catalog_manager.cc:4841] TS 4109ccbef0b043068afd24ef7bdec85e (127.15.77.67:39975): tablet ae6002a4f2504a83a3b66a531962a165 (table test-table [id=af25838e6228406085f58e2f437fc1fb]) successfully deleted
I20260620 18:44:58.380338 23222 catalog_manager.cc:4841] TS f30f957fba8444739c7c1f7b3fe41f02 (127.15.77.66:37863): tablet ae6002a4f2504a83a3b66a531962a165 (table test-table [id=af25838e6228406085f58e2f437fc1fb]) successfully deleted
W20260620 18:44:58.384496 23220 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:33647 (error 108)
I20260620 18:44:58.385118 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:44:58.384927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33647 (local address 127.0.0.1:43514)
0620 18:44:58.385047 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 18:44:58.385085 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:33647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260620 18:44:58.385120 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 23448
W20260620 18:44:58.385232 23220 catalog_manager.cc:4584] TS bdfd69d052cf428ea35e6e4b4d7abb7a (127.15.77.65:33647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae6002a4f2504a83a3b66a531962a165: Network error: Client connection negotiation failed: client connection to 127.15.77.65:33647: connect: Connection refused (error 111)
I20260620 18:44:58.391631 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 23588
I20260620 18:44:58.397830 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 23204
2026-06-20T18:44:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3094 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:44:58 dist-test-slave-tvl9 krb5kdc[23859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:44:58 dist-test-slave-tvl9 krb5kdc[23859](info): set up 2 sockets
Jun 20 18:44:58 dist-test-slave-tvl9 krb5kdc[23859](info): commencing operation
krb5kdc: starting...
W20260620 18:44:59.455706 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:44:59 dist-test-slave-tvl9 krb5kdc[23859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:44:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:44:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:44:59.574131 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:41683
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:45027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:41683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:44:59.656510 23872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:44:59.656767 23872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:44:59.656806 23872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:44:59.658004 23872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:44:59.658070 23872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:44:59.658093 23872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:44:59.658114 23872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:44:59.658145 23872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:44:59.659714 23872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:41683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:41683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:44:59.660001 23872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:44:59.660239 23872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:44:59.665222 23872 server_base.cc:1034] running on GCE node
W20260620 18:44:59.665185 23877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:59.665149 23878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:44:59.665158 23880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:44:59.665614 23872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:44:59.665908 23872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:44:59.667055 23872 hybrid_clock.cc:648] HybridClock initialized: now 1781981099667036 us; error 45 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-20T18:44:59Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1273 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:44:59 dist-test-slave-tvl9 krb5kdc[23885](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:44:59 dist-test-slave-tvl9 krb5kdc[23885](info): set up 2 sockets
Jun 20 18:44:59 dist-test-slave-tvl9 krb5kdc[23885](info): commencing operation
krb5kdc: starting...
W20260620 18:45:00.711850 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:45:00 dist-test-slave-tvl9 krb5kdc[23885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:45:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:45:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:45:00.834188 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:38719
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:38719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:45:00.916700 23898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:00.916940 23898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:00.916965 23898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:00.918201 23898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:45:00.918253 23898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:00.918267 23898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:00.918280 23898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:45:00.918340 23898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:45:00.919946 23898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:38719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:38719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:00.920234 23898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:00.920487 23898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:00.926303 23904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:00.926309 23903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:00.926532 23898 server_base.cc:1034] running on GCE node
W20260620 18:45:00.926350 23906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:00.926847 23898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:00.927120 23898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:00.928267 23898 hybrid_clock.cc:648] HybridClock initialized: now 1781981100928258 us; error 32 us; skew 500 ppm
Jun 20 18:45:00 dist-test-slave-tvl9 krb5kdc[23885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981100, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:00.934530 23898 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:45:00.935067 23898 webserver.cc:460] Webserver started at http://127.15.77.126:41161/ using document root <none> and password file <none>
I20260620 18:45:00.935387 23898 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:00.935460 23898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:00.935626 23898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:00.936734 23898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "5600975a3be64b8fb343787156620545"
format_stamp: "Formatted at 2026-06-20 18:45:00 on dist-test-slave-tvl9"
server_key: "f5cea36414ff0c4d436aa539b2c4e101"
server_key_iv: "e215fcc9ee7d8bb3b4211777517cc23c"
server_key_version: "encryptionkey@0"
I20260620 18:45:00.937109 23898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "5600975a3be64b8fb343787156620545"
format_stamp: "Formatted at 2026-06-20 18:45:00 on dist-test-slave-tvl9"
server_key: "f5cea36414ff0c4d436aa539b2c4e101"
server_key_iv: "e215fcc9ee7d8bb3b4211777517cc23c"
server_key_version: "encryptionkey@0"
I20260620 18:45:00.938506 23898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:45:00.939255 23913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:00.939498 23898 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:45:00.939600 23898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "5600975a3be64b8fb343787156620545"
format_stamp: "Formatted at 2026-06-20 18:45:00 on dist-test-slave-tvl9"
server_key: "f5cea36414ff0c4d436aa539b2c4e101"
server_key_iv: "e215fcc9ee7d8bb3b4211777517cc23c"
server_key_version: "encryptionkey@0"
I20260620 18:45:00.939692 23898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:00.962980 23898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:00.965317 23898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:00.965515 23898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:00.970057 23898 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:38719
I20260620 18:45:00.970108 23965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:38719 every 8 connection(s)
I20260620 18:45:00.970515 23898 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:45:00.971473 23966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:45:00.973824 23966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545: Bootstrap starting.
I20260620 18:45:00.974428 23966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545: Neither blocks nor log segments found. Creating new log.
I20260620 18:45:00.974741 23966 log.cc:824] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545: Log is configured to *not* fsync() on all Append() calls
I20260620 18:45:00.975553 23966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545: No bootstrap required, opened a new log
I20260620 18:45:00.977174 23966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } }
I20260620 18:45:00.977319 23966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:45:00.977342 23966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5600975a3be64b8fb343787156620545, State: Initialized, Role: FOLLOWER
I20260620 18:45:00.977522 23966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } }
I20260620 18:45:00.977588 23966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:45:00.977610 23966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:45:00.977685 23966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:45:00.978238 23966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } }
I20260620 18:45:00.978351 23966 leader_election.cc:302] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5600975a3be64b8fb343787156620545; no voters:
I20260620 18:45:00.978534 23966 leader_election.cc:288] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:45:00.978567 23969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:45:00.978760 23969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 1 LEADER]: Becoming Leader. State: Replica: 5600975a3be64b8fb343787156620545, State: Running, Role: LEADER
I20260620 18:45:00.978818 23966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:45:00.978850 23969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } }
I20260620 18:45:00.979274 23970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5600975a3be64b8fb343787156620545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } } }
I20260620 18:45:00.979463 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 23898
I20260620 18:45:00.979478 23970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [sys.catalog]: This master's current role is: LEADER
I20260620 18:45:00.979593 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:45:00.979796 15669 external_mini_cluster.cc:1428] Setting key dfe4894e3ed5266769408f1398eecb2b
I20260620 18:45:00.980090 23973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:45:00.980211 23971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5600975a3be64b8fb343787156620545. Latest consensus state: current_term: 1 leader_uuid: "5600975a3be64b8fb343787156620545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } } }
I20260620 18:45:00.980297 23971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [sys.catalog]: This master's current role is: LEADER
I20260620 18:45:00.980759 23973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:45:00.983588 23973 catalog_manager.cc:1347] Generated new cluster ID: a99e79fec5b84583817223028191e8a1
I20260620 18:45:00.983693 23973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:45:00 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:45:00.992193 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:00.980458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60426 (local address 127.15.77.126:38719)
0620 18:45:00.980858 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:00.980867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:00.980901 (+ 34us) server_negotiation.cc:407] Connection header received
0620 18:45:00.981031 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:00.981038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:00.981106 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:00.981283 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:00.982138 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:00.983402 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:00.984532 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:00.985189 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:00.985317 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:00.988286 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:00.988291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:00.988292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:45:00.988305 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:00.990394 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:00.990835 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:00.990838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:00.990839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:00.990949 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:00.991163 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:00.991164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:00.991164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:00.991522 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:00.991663 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:00.991808 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 18:45:00.991943 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":77,"threads_started":1}
I20260620 18:45:00.999413 23973 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:45:01.000151 23973 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:45:01.007706 23973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545: Generated new TSK 0
I20260620 18:45:01.007931 23973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:45:01.038509 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38719
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:45:01.124728 23992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:01.124986 23992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:01.125030 23992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:01.126260 23992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:01.126333 23992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:01.126379 23992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:45:01.127978 23992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:01.128263 23992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:01.128505 23992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:01.133777 23997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:01.133783 23998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.133922 23992 server_base.cc:1034] running on GCE node
W20260620 18:45:01.133785 24000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.134245 23992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:01.134466 23992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:01.135602 23992 hybrid_clock.cc:648] HybridClock initialized: now 1781981101135600 us; error 44 us; skew 500 ppm
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:01.138165 23992 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:45:01.138609 23992 webserver.cc:460] Webserver started at http://127.15.77.65:42581/ using document root <none> and password file <none>
I20260620 18:45:01.138859 23992 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:01.138906 23992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:01.139071 23992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:01.140244 23992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "23f4d82f0d2b4886a0a525e81ea59e77"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "03686c0717009309fe0288fd6d4a6372"
server_key_iv: "6adc80b92c07d66a464b3f5106f1a411"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.140606 23992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "23f4d82f0d2b4886a0a525e81ea59e77"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "03686c0717009309fe0288fd6d4a6372"
server_key_iv: "6adc80b92c07d66a464b3f5106f1a411"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.142025 23992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:45:01.142846 24007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.143049 23992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:45:01.143146 23992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "23f4d82f0d2b4886a0a525e81ea59e77"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "03686c0717009309fe0288fd6d4a6372"
server_key_iv: "6adc80b92c07d66a464b3f5106f1a411"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.143216 23992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:01.162379 23992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:01.164386 23992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:01.164500 23992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:01.164700 23992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:01.165037 23992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:01.165074 23992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.165123 23992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:01.165153 23992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.170955 23992 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:44075
I20260620 18:45:01.171039 24120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:44075 every 8 connection(s)
I20260620 18:45:01.171455 23992 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:45:01.174302 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 23992
I20260620 18:45:01.174434 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:45:01.174618 15669 external_mini_cluster.cc:1428] Setting key 2942462d3d2ab923d428a2d747604958
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:45:01.182189 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:01.172255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:49455 (local address 127.15.77.126:38719)
0620 18:45:01.172309 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:01.172312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:01.172554 (+ 242us) server_negotiation.cc:407] Connection header received
0620 18:45:01.173035 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:01.173036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:01.173047 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:01.173113 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:01.174102 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.174708 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.175938 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.176167 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.176257 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.178808 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:01.178810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:01.178810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:45:01.178817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:01.180510 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:01.181155 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:01.181156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:01.181156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:01.181212 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:01.181507 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:01.181508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:01.181508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:01.181615 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:01.181664 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:01.182029 (+ 365us) server_negotiation.cc:299] Negotiation successful
0620 18:45:01.182120 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:45:01.182485 24123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:01.172239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:49455)
0620 18:45:01.172412 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:01.172450 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:01.172937 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:01.173193 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:01.173197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:01.173788 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:01.173989 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.173994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.174796 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.174797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.175842 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.175844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.176271 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.176272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.176406 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.176969 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:01.176979 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:01.178696 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:01.180642 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:01.180652 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:01.180656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:01.181071 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:01.181282 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:01.181283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:01.181283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:01.181434 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:01.181679 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:01.181687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:01.181877 (+ 190us) client_negotiation.cc:769] Sending connection context
0620 18:45:01.182012 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 18:45:01.182208 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260620 18:45:01.183156 24121 heartbeater.cc:347] Connected to a master server at 127.15.77.126:38719
I20260620 18:45:01.183252 24121 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:45:01.183478 24121 heartbeater.cc:511] Master 127.15.77.126:38719 requested a full tablet report, sending...
I20260620 18:45:01.184070 23930 ts_manager.cc:194] Registered new tserver with Master: 23f4d82f0d2b4886a0a525e81ea59e77 (127.15.77.65:44075)
I20260620 18:45:01.184746 23930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:49455
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:45:01.200433 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38719
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:45:01.286062 24126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:01.286278 24126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:01.286303 24126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:01.287513 24126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:01.287557 24126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:01.287585 24126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:45:01.288986 24126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:01.289184 24126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:01.289369 24126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:01.294555 24132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.294734 24126 server_base.cc:1034] running on GCE node
W20260620 18:45:01.294579 24131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:01.294555 24134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.295043 24126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:01.295348 24126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:01.296515 24126 hybrid_clock.cc:648] HybridClock initialized: now 1781981101296508 us; error 33 us; skew 500 ppm
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:01.299016 24126 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:45:01.299589 24126 webserver.cc:460] Webserver started at http://127.15.77.66:41329/ using document root <none> and password file <none>
I20260620 18:45:01.299853 24126 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:01.299921 24126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:01.300077 24126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:01.301131 24126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "fd24e6481b8e4cf187d9b1484a1d3d89"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "a55d1d9e95de0aad517b9480bf7823c6"
server_key_iv: "c9ec6de9b942ee68ddcefc76c3bbeacf"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.301505 24126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "fd24e6481b8e4cf187d9b1484a1d3d89"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "a55d1d9e95de0aad517b9480bf7823c6"
server_key_iv: "c9ec6de9b942ee68ddcefc76c3bbeacf"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.302839 24126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:45:01.303647 24141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.303820 24126 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:45:01.303929 24126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "fd24e6481b8e4cf187d9b1484a1d3d89"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "a55d1d9e95de0aad517b9480bf7823c6"
server_key_iv: "c9ec6de9b942ee68ddcefc76c3bbeacf"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.304004 24126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:01.316756 24126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:01.318783 24126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:01.318957 24126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:01.319206 24126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:01.319640 24126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:01.319700 24126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.319742 24126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:01.319790 24126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.326050 24126 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:40785
I20260620 18:45:01.326088 24254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:40785 every 8 connection(s)
I20260620 18:45:01.326442 24126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:45:01.335664 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 24126
I20260620 18:45:01.335788 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:45:01.335949 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:01.327204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:54875 (local address 127.15.77.126:38719)
0620 18:45:01.327252 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:01.327256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:01.327493 (+ 237us) server_negotiation.cc:407] Connection header received
0620 18:45:01.327974 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:01.327975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:01.327987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:01.328058 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:01.328983 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.329485 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.330681 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.330884 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.330943 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.333085 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:01.333087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:01.333087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:45:01.333094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:01.334586 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:01.334944 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:01.334944 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:01.334945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:01.334998 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:01.335222 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:01.335223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:01.335223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:01.335398 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:01.335448 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:01.335754 (+ 306us) server_negotiation.cc:299] Negotiation successful
0620 18:45:01.335855 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:45:01.335963 15669 external_mini_cluster.cc:1428] Setting key 8f7737b4bff420877b51beaa955209ec
I20260620 18:45:01.336150 24257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:01.327154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:54875)
0620 18:45:01.327355 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:01.327399 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:01.327884 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:01.328116 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:01.328119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:01.328699 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:01.328902 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.328906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.329575 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.329576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.330574 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.330577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.330964 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.330964 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.331068 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.331610 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:01.331619 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:01.332992 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:01.334658 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:01.334663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:01.334665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:01.334878 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:01.335057 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:01.335058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:01.335058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:01.335157 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:01.335453 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:01.335458 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:01.335612 (+ 154us) client_negotiation.cc:769] Sending connection context
0620 18:45:01.335726 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 18:45:01.335891 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260620 18:45:01.336791 24255 heartbeater.cc:347] Connected to a master server at 127.15.77.126:38719
I20260620 18:45:01.336912 24255 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:01.337152 24255 heartbeater.cc:511] Master 127.15.77.126:38719 requested a full tablet report, sending...
I20260620 18:45:01.337559 23930 ts_manager.cc:194] Registered new tserver with Master: fd24e6481b8e4cf187d9b1484a1d3d89 (127.15.77.66:40785)
I20260620 18:45:01.337944 23930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:54875
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:45:01.362356 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38719
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:45:01.445639 24260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:01.445914 24260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:01.445955 24260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:01.447250 24260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:01.447322 24260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:01.447402 24260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:45:01.448918 24260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:38719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:01.449201 24260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:01.449440 24260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:01.454531 24266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:01.454596 24265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.454857 24260 server_base.cc:1034] running on GCE node
W20260620 18:45:01.454926 24268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.455142 24260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:01.455448 24260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:01.456684 24260 hybrid_clock.cc:648] HybridClock initialized: now 1781981101456651 us; error 26 us; skew 500 ppm
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:01.459210 24260 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:45:01.459715 24260 webserver.cc:460] Webserver started at http://127.15.77.67:40165/ using document root <none> and password file <none>
I20260620 18:45:01.459956 24260 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:01.460026 24260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:01.460186 24260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:01.461251 24260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "a5a3f36034db4cebaa41bd3caf71fe9d"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "d7fc0718aac8b8920dbb57ad76cea7f6"
server_key_iv: "ef16f34e8323c43c184cc96c6693e1ee"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.461616 24260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "a5a3f36034db4cebaa41bd3caf71fe9d"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "d7fc0718aac8b8920dbb57ad76cea7f6"
server_key_iv: "ef16f34e8323c43c184cc96c6693e1ee"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.462965 24260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:45:01.463798 24275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.463973 24260 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.464082 24260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "a5a3f36034db4cebaa41bd3caf71fe9d"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "d7fc0718aac8b8920dbb57ad76cea7f6"
server_key_iv: "ef16f34e8323c43c184cc96c6693e1ee"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.464156 24260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:01.471890 24260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:01.473780 24260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:01.473948 24260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:01.474182 24260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:01.474503 24260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:01.474558 24260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.474601 24260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:01.474644 24260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.480966 24260 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:34895
I20260620 18:45:01.480999 24388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:34895 every 8 connection(s)
I20260620 18:45:01.481432 24260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:45:01.487427 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 24260
I20260620 18:45:01.487558 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:45:01.487773 15669 external_mini_cluster.cc:1428] Setting key fdd62d3280e292b827917d875ce48ddc
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:45:01.491119 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:01.482156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:40737 (local address 127.15.77.126:38719)
0620 18:45:01.482237 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:01.482245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:01.482447 (+ 202us) server_negotiation.cc:407] Connection header received
0620 18:45:01.482995 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:01.482996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:01.483010 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:01.483069 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:01.484143 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.484648 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.485692 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.485887 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.485980 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.488387 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:01.488388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:01.488389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:45:01.488397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:01.489633 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:01.490086 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:01.490087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:01.490088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:01.490156 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:01.490429 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:01.490430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:01.490431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:01.490561 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:01.490635 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:01.490935 (+ 300us) server_negotiation.cc:299] Negotiation successful
0620 18:45:01.491022 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 18:45:01.491452 24391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:01.482171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:40737)
0620 18:45:01.482317 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:01.482356 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:01.482884 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:01.483149 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:01.483152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:01.483819 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:01.484026 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.484030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.484758 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.484760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.485567 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.485568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.485994 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.485995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.486122 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.486619 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:01.486628 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:01.488288 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:01.489740 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:01.489748 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:01.489750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:01.489990 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:01.490228 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:01.490229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:01.490229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:01.490353 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:01.490640 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:01.490647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:01.490798 (+ 151us) client_negotiation.cc:769] Sending connection context
0620 18:45:01.490910 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 18:45:01.491084 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260620 18:45:01.491988 24389 heartbeater.cc:347] Connected to a master server at 127.15.77.126:38719
I20260620 18:45:01.492098 24389 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:01.492311 24389 heartbeater.cc:511] Master 127.15.77.126:38719 requested a full tablet report, sending...
I20260620 18:45:01.492693 23930 ts_manager.cc:194] Registered new tserver with Master: a5a3f36034db4cebaa41bd3caf71fe9d (127.15.77.67:34895)
I20260620 18:45:01.493086 23930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:40737
I20260620 18:45:01.499984 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab.
Entry for principal oryx/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab.
WARNING: no policy specified for oryx/127.15.77.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab.
Entry for principal oryx/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab.
WARNING: no policy specified for oryx/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab.
Entry for principal oryx/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab.
I20260620 18:45:01.573694 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 23992
I20260620 18:45:01.578543 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 24126
I20260620 18:45:01.583103 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 24260
I20260620 18:45:01.587509 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 23898
I20260620 18:45:01.593029 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:38719
--webserver_interface=127.15.77.126
--webserver_port=41161
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:38719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:45:01.676091 24398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:01.676360 24398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:01.676398 24398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:01.677611 24398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:45:01.677712 24398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:01.677736 24398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:01.677755 24398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:45:01.677783 24398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:45:01.679401 24398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:38719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:38719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=41161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:01.679723 24398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:01.679972 24398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:01.685134 24404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.685324 24398 server_base.cc:1034] running on GCE node
W20260620 18:45:01.685083 24403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:01.685091 24406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.685753 24398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:01.686085 24398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:01.687276 24398 hybrid_clock.cc:648] HybridClock initialized: now 1781981101687262 us; error 59 us; skew 500 ppm
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:01.690109 24398 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:45:01.690573 24398 webserver.cc:460] Webserver started at http://127.15.77.126:41161/ using document root <none> and password file <none>
I20260620 18:45:01.690855 24398 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:01.690941 24398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:01.692399 24398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:45:01.693151 24413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.693346 24398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:45:01.693456 24398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "5600975a3be64b8fb343787156620545"
format_stamp: "Formatted at 2026-06-20 18:45:00 on dist-test-slave-tvl9"
server_key: "f5cea36414ff0c4d436aa539b2c4e101"
server_key_iv: "e215fcc9ee7d8bb3b4211777517cc23c"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.693841 24398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:01.705250 24398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:01.707906 24398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:01.708081 24398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:01.712414 24398 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:38719
I20260620 18:45:01.712461 24465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:38719 every 8 connection(s)
I20260620 18:45:01.712909 24398 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:45:01.714442 24466 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:45:01.715160 24466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545: Bootstrap starting.
I20260620 18:45:01.718062 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 24398
I20260620 18:45:01.718420 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:44075
--local_ip_for_outbound_sockets=127.15.77.65
--tserver_master_addrs=127.15.77.126:38719
--webserver_port=42581
--webserver_interface=127.15.77.65
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.77.65
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:45:01.725708 24466 log.cc:824] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545: Log is configured to *not* fsync() on all Append() calls
I20260620 18:45:01.728315 24466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:45:01.728684 24466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545: Bootstrap complete.
I20260620 18:45:01.732332 24466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } }
I20260620 18:45:01.732668 24466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5600975a3be64b8fb343787156620545, State: Initialized, Role: FOLLOWER
I20260620 18:45:01.732882 24466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } }
I20260620 18:45:01.732955 24466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:45:01.733016 24466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:45:01.733062 24466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 1 FOLLOWER]: Advancing to term 2
I20260620 18:45:01.734459 24466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } }
I20260620 18:45:01.734579 24466 leader_election.cc:302] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5600975a3be64b8fb343787156620545; no voters:
I20260620 18:45:01.734753 24466 leader_election.cc:288] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 18:45:01.734850 24470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 18:45:01.735019 24466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:45:01.735069 24470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [term 2 LEADER]: Becoming Leader. State: Replica: 5600975a3be64b8fb343787156620545, State: Running, Role: LEADER
I20260620 18:45:01.735224 24470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } }
I20260620 18:45:01.735627 24471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5600975a3be64b8fb343787156620545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } } }
I20260620 18:45:01.735718 24471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [sys.catalog]: This master's current role is: LEADER
I20260620 18:45:01.736047 24472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5600975a3be64b8fb343787156620545. Latest consensus state: current_term: 2 leader_uuid: "5600975a3be64b8fb343787156620545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5600975a3be64b8fb343787156620545" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 38719 } } }
I20260620 18:45:01.736116 24472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545 [sys.catalog]: This master's current role is: LEADER
I20260620 18:45:01.736120 24479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:45:01.736814 24479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:45:01.736948 24479 catalog_manager.cc:1259] Loaded cluster ID: a99e79fec5b84583817223028191e8a1
I20260620 18:45:01.736984 24479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:45:01.738125 24479 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:45:01.738322 24479 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5600975a3be64b8fb343787156620545: Loaded TSK: 0
I20260620 18:45:01.738554 24479 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 18:45:01.807873 24468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:01.808122 24468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:01.808161 24468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:01.809363 24468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:01.809432 24468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:01.809476 24468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:45:01.810957 24468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.65.keytab
--principal=oryx/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:44075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=42581
--tserver_master_addrs=127.15.77.126:38719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:01.811232 24468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:01.811501 24468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:01.816536 24492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:01.816541 24495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:01.816555 24493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.817022 24468 server_base.cc:1034] running on GCE node
I20260620 18:45:01.817265 24468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:01.817590 24468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:01.818763 24468 hybrid_clock.cc:648] HybridClock initialized: now 1781981101818744 us; error 33 us; skew 500 ppm
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981101, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:01.821884 24468 init.cc:377] Logged in from keytab as oryx/127.15.77.65@KRBTEST.COM (short username oryx)
I20260620 18:45:01.822435 24468 webserver.cc:460] Webserver started at http://127.15.77.65:42581/ using document root <none> and password file <none>
I20260620 18:45:01.822713 24468 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:01.822799 24468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:01.824656 24468 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:45:01.825374 24502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.825582 24468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:45:01.825685 24468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "23f4d82f0d2b4886a0a525e81ea59e77"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "03686c0717009309fe0288fd6d4a6372"
server_key_iv: "6adc80b92c07d66a464b3f5106f1a411"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.826133 24468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:01.859704 24468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:01.862244 24468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:01.862392 24468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:01.862650 24468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:01.862985 24468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:01.863024 24468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.863054 24468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:01.863072 24468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.870141 24468 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:44075
I20260620 18:45:01.870235 24615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:44075 every 8 connection(s)
I20260620 18:45:01.870590 24468 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:45:01.875129 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 24468
I20260620 18:45:01.875463 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:40785
--local_ip_for_outbound_sockets=127.15.77.66
--tserver_master_addrs=127.15.77.126:38719
--webserver_port=41329
--webserver_interface=127.15.77.66
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.77.66
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:01.880206 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:01.871381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:34423)
0620 18:45:01.871573 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:01.871625 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:01.872156 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:01.872616 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:01.872620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:01.873200 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:01.873394 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.873399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.874492 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.874493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.875625 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.875628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.876474 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.876475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.876636 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.877273 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:01.877283 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:01.880001 (+ 2718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":78,"threads_started":1}
W20260620 18:45:01.880424 24619 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:01.871446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:34423 (local address 127.15.77.126:38719)
0620 18:45:01.871712 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:01.871720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:01.871757 (+ 37us) server_negotiation.cc:407] Connection header received
0620 18:45:01.872388 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:01.872393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:01.872437 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:01.872574 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:01.873520 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.874385 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.875722 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.876373 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.876493 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.880193 (+ 3700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:34423: BlockingRecv error: recv got EOF from 127.15.77.65:34423 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":108,"threads_started":1}
W20260620 18:45:01.880798 24616 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:38719 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.77.126:38719: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:01.886001 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:01.881067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:42253)
0620 18:45:01.881158 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:01.881172 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:01.881225 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:01.881390 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:01.881391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:01.881525 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:01.881578 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.881580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.882561 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.882562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.883070 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.883072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.883615 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.883616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.883681 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.884163 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:01.884168 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:01.885955 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
W20260620 18:45:01.886169 24619 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:01.881116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:42253 (local address 127.15.77.126:38719)
0620 18:45:01.881168 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:01.881171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:01.881214 (+ 43us) server_negotiation.cc:407] Connection header received
0620 18:45:01.881307 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:01.881308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:01.881322 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:01.881388 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:01.881655 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.882474 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.883163 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.883537 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.883620 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.886075 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:42253: BlockingRecv error: recv got EOF from 127.15.77.65:42253 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:01.890640 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:01.886311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:34671)
0620 18:45:01.886393 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:01.886401 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:01.886429 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:01.886593 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:01.886593 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:01.886701 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:01.886727 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.886728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.887672 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.887672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.888140 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:01.888141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.888580 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.888581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:01.888616 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.889026 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:01.889031 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:01.890595 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260620 18:45:01.890755 24619 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:01.886427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:34671 (local address 127.15.77.126:38719)
0620 18:45:01.886471 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:01.886473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:01.886482 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:45:01.886504 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:01.886505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:01.886516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:01.886584 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:01.886802 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.887601 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.888221 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:01.888514 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:01.888577 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:01.890695 (+ 2118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:34671: BlockingRecv error: recv got EOF from 127.15.77.65:34671 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 18:45:01.890861 24616 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 18:45:01.963573 24620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:01.963841 24620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:01.963866 24620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:01.964990 24620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:01.965025 24620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:01.965052 24620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:45:01.966411 24620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.66.keytab
--principal=oryx/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:40785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=41329
--tserver_master_addrs=127.15.77.126:38719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:01.966615 24620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:01.966800 24620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:01.972204 24626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:01.972215 24625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.972473 24620 server_base.cc:1034] running on GCE node
W20260620 18:45:01.972215 24628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:01.972790 24620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:01.973057 24620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:01.974221 24620 hybrid_clock.cc:648] HybridClock initialized: now 1781981101974217 us; error 36 us; skew 500 ppm
Jun 20 18:45:01 dist-test-slave-tvl9 krb5kdc[23885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981101, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:01.976917 24620 init.cc:377] Logged in from keytab as oryx/127.15.77.66@KRBTEST.COM (short username oryx)
I20260620 18:45:01.977387 24620 webserver.cc:460] Webserver started at http://127.15.77.66:41329/ using document root <none> and password file <none>
I20260620 18:45:01.977581 24620 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:01.977639 24620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:01.978927 24620 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:45:01.979550 24635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.979719 24620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:45:01.979807 24620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "fd24e6481b8e4cf187d9b1484a1d3d89"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "a55d1d9e95de0aad517b9480bf7823c6"
server_key_iv: "c9ec6de9b942ee68ddcefc76c3bbeacf"
server_key_version: "encryptionkey@0"
I20260620 18:45:01.980142 24620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:01.988401 24620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:01.990258 24620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:01.990413 24620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:01.990648 24620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:01.991008 24620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:01.991063 24620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.991104 24620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:01.991130 24620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:01.997041 24620 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:40785
I20260620 18:45:01.997151 24748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:40785 every 8 connection(s)
I20260620 18:45:01.997506 24620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:45:02.001150 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 24620
I20260620 18:45:02.001466 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:34895
--local_ip_for_outbound_sockets=127.15.77.67
--tserver_master_addrs=127.15.77.126:38719
--webserver_port=40165
--webserver_interface=127.15.77.67
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.77.67
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:02.005856 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:01.998290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:37383)
0620 18:45:01.998447 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:01.998486 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:01.998970 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:01.999187 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:01.999190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:01.999865 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:02.000065 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.000071 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.000745 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.000746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.001764 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.001767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.002300 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.002301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.002413 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.003026 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:02.003038 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:02.005686 (+ 2648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
W20260620 18:45:02.005930 24619 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:01.998292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:37383 (local address 127.15.77.126:38719)
0620 18:45:01.998365 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:01.998367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:01.998556 (+ 189us) server_negotiation.cc:407] Connection header received
0620 18:45:01.999059 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:01.999060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:01.999074 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:01.999140 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:02.000166 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.000659 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.001866 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.002206 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.002285 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.005838 (+ 3553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:37383: BlockingRecv error: recv got EOF from 127.15.77.66:37383 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260620 18:45:02.006283 24749 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:38719 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.77.126:38719: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:02.011070 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:02.006549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:60229)
0620 18:45:02.006605 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:02.006616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:02.006692 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:02.006848 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:02.006849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:02.006988 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:02.007036 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.007038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.007769 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.007770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.008281 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.008283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.008656 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.008657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.008725 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.009192 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:02.009196 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:02.010959 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 18:45:02.011101 24619 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:02.006619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:60229 (local address 127.15.77.126:38719)
0620 18:45:02.006687 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:02.006689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:02.006699 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:45:02.006763 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:02.006764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:02.006774 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:02.006843 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:02.007135 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.007690 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.008375 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.008566 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.008638 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.011052 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:60229: BlockingRecv error: recv got EOF from 127.15.77.66:60229 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:02.015750 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:02.011470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:37427)
0620 18:45:02.011524 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:02.011535 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:02.011594 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:02.011737 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:02.011738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:02.011870 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:02.011901 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.011903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.012537 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.012538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.013057 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.013058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.013439 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.013440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.013491 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.013940 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:02.013945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:02.015672 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 18:45:02.015848 24619 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:02.011552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:37427 (local address 127.15.77.126:38719)
0620 18:45:02.011612 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:02.011614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:02.011622 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:45:02.011656 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:02.011657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:02.011666 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:02.011734 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:02.011984 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.012455 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.013149 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.013339 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.013440 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.015762 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:37427: BlockingRecv error: recv got EOF from 127.15.77.66:37427 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260620 18:45:02.016093 24749 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 18:45:02.087144 24752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:02.087437 24752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:02.087468 24752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:02.088670 24752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:02.088716 24752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:02.088743 24752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:45:02.090178 24752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/krb5kdc/oryx_127.15.77.67.keytab
--principal=oryx/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:34895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=40165
--tserver_master_addrs=127.15.77.126:38719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:02.090387 24752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:02.090588 24752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:02.096132 24757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:02.096148 24758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:02.096268 24752 server_base.cc:1034] running on GCE node
W20260620 18:45:02.096165 24760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:02.096743 24752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:02.097153 24752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:02.098351 24752 hybrid_clock.cc:648] HybridClock initialized: now 1781981102098327 us; error 42 us; skew 500 ppm
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981102, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:02.101131 24752 init.cc:377] Logged in from keytab as oryx/127.15.77.67@KRBTEST.COM (short username oryx)
I20260620 18:45:02.101562 24752 webserver.cc:460] Webserver started at http://127.15.77.67:40165/ using document root <none> and password file <none>
I20260620 18:45:02.101773 24752 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:02.101825 24752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:02.103183 24752 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:45:02.103835 24767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:02.104012 24752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:02.104100 24752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "a5a3f36034db4cebaa41bd3caf71fe9d"
format_stamp: "Formatted at 2026-06-20 18:45:01 on dist-test-slave-tvl9"
server_key: "d7fc0718aac8b8920dbb57ad76cea7f6"
server_key_iv: "ef16f34e8323c43c184cc96c6693e1ee"
server_key_version: "encryptionkey@0"
I20260620 18:45:02.104445 24752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:02.128753 24752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:02.131384 24752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:02.131547 24752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:02.131788 24752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:02.132212 24752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:02.132269 24752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:02.132314 24752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:02.132340 24752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:02.138520 24752 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:34895
I20260620 18:45:02.138558 24880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:34895 every 8 connection(s)
I20260620 18:45:02.138973 24752 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
W20260620 18:45:02.147058 24619 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:02.139843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:51891 (local address 127.15.77.126:38719)
0620 18:45:02.139895 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:02.139897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:02.140149 (+ 252us) server_negotiation.cc:407] Connection header received
0620 18:45:02.140722 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:02.140724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:02.140736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:02.140796 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:02.141758 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.142261 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.143277 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.143534 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.143627 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.146997 (+ 3370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:51891: BlockingRecv error: recv got EOF from 127.15.77.67:51891 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:45:02.147141 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:02.139845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:51891)
0620 18:45:02.140022 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:02.140059 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:02.140606 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:02.140876 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:02.140880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:02.141451 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:02.141649 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.141655 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.142361 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.142363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.143179 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.143181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.143642 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.143643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.143777 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.144344 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:02.144353 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:02.146923 (+ 2570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
W20260620 18:45:02.147537 24881 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:38719 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.77.126:38719: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
I20260620 18:45:02.147540 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 24752
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:02.152626 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:02.147803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:52473)
0620 18:45:02.147861 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:02.147875 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:02.147923 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:02.148152 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:02.148153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:02.148316 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:02.148384 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.148386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.149068 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.149069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.149660 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.149662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.149990 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.149991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.150068 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.150516 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:02.150521 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:02.152491 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 18:45:02.152693 24619 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:02.147970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:52473 (local address 127.15.77.126:38719)
0620 18:45:02.148011 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:02.148013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:02.148021 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:45:02.148046 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:02.148047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:02.148057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:02.148124 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:02.148460 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.148972 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.149740 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.149916 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.149966 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.152618 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:52473: BlockingRecv error: recv got EOF from 127.15.77.67:52473 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:45:02.155555 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:02.147984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34062 (local address 127.15.77.126:38719)
0620 18:45:02.148092 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:02.148094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:02.148150 (+ 56us) server_negotiation.cc:407] Connection header received
0620 18:45:02.148227 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:02.148228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:02.148235 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:02.148281 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:02.148917 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.149423 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.150462 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.150666 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.150719 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.152058 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:02.152064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:02.152065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:45:02.152081 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:02.154015 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:02.154336 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:02.154340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:02.154341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:02.154468 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:02.154684 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:02.154685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:02.154685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:02.154974 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:02.155136 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:02.155268 (+ 132us) server_negotiation.cc:299] Negotiation successful
0620 18:45:02.155429 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:02.157370 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:02.152941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:59655)
0620 18:45:02.152992 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:02.153001 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:02.153039 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:02.153198 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:02.153199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:02.153345 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:02.153378 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.153380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.154007 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.154008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.154718 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.154721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.155037 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.155038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.155088 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.155538 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:02.155543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:02.157322 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 18:45:02.157497 24619 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:02.153029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:59655 (local address 127.15.77.126:38719)
0620 18:45:02.153072 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:02.153074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:02.153083 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:45:02.153104 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:02.153105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:02.153114 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:02.153178 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:02.153453 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.153925 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.154790 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.154966 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.155013 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.157422 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:59655: BlockingRecv error: recv got EOF from 127.15.77.67:59655 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 18:45:02.157502 24881 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:02 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:02.895807 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:02.891286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:44703)
0620 18:45:02.891463 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:02.891474 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:02.891528 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:02.891674 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:02.891675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:02.891834 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:02.891884 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.891885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.892582 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.892583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.893142 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:02.893144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.893507 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.893508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:02.893548 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.893981 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:02.893985 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:02.895738 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":112,"threads_started":1}
W20260620 18:45:02.895920 24887 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:02.891408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:44703 (local address 127.15.77.126:38719)
0620 18:45:02.891542 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:02.891545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:02.891561 (+ 16us) server_negotiation.cc:407] Connection header received
0620 18:45:02.891584 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:02.891584 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:02.891594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:02.891657 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:02.891946 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.892502 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.893228 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:02.893433 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:02.893497 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:02.895840 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:44703: BlockingRecv error: recv got EOF from 127.15.77.65:44703 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":92,"threads_started":1}
Jun 20 18:45:03 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:03 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:03.021581 24888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:03.016466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:44423)
0620 18:45:03.016653 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:03.016667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:03.016768 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:03.016934 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:03.016935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:03.017112 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:03.017161 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:03.017176 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.017885 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.017886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:03.018446 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:03.018447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.018852 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.018853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:03.018920 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:03.019534 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:03.019538 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:03.021465 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":90,"threads_started":1}
W20260620 18:45:03.021605 24887 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:03.016546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:44423 (local address 127.15.77.126:38719)
0620 18:45:03.016655 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:03.016657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:03.016753 (+ 96us) server_negotiation.cc:407] Connection header received
0620 18:45:03.016854 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:03.016855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:03.016867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:03.016923 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:03.017269 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.017792 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.018544 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.018760 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.018850 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:03.021529 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:44423: BlockingRecv error: recv got EOF from 127.15.77.66:44423 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 20 18:45:03 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:03 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:03.163126 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:03.158048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:39199)
0620 18:45:03.158400 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:03.158413 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:03.158515 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:03.158710 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:03.158711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:03.158888 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:03.158935 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:03.158937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.159686 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.159687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:03.160236 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:03.160238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.160625 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.160626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:03.160697 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:03.161141 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:03.161145 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:03.163017 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":139,"threads_started":1}
W20260620 18:45:03.163187 24887 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:03.158175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:39199 (local address 127.15.77.126:38719)
0620 18:45:03.158384 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:03.158387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:03.158498 (+ 111us) server_negotiation.cc:407] Connection header received
0620 18:45:03.158594 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:03.158595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:03.158610 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:03.158688 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:03.159018 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.159599 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.160342 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.160536 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.160625 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:03.163098 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:39199: BlockingRecv error: recv got EOF from 127.15.77.67:39199 (error 108)
Metrics: {"server-negotiator.queue_time_us":174}
Jun 20 18:45:03 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:03 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:03.901458 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:03.896395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:53509)
0620 18:45:03.896538 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:03.896552 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:03.896620 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:03.897023 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:03.897024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:03.897184 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:03.897238 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:03.897240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.897988 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.897989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:03.898567 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:03.898570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.898966 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.898967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:03.899029 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:03.899524 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:03.899529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:03.901396 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
W20260620 18:45:03.901621 24891 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:03.896543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:53509 (local address 127.15.77.126:38719)
0620 18:45:03.896823 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:03.896826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:03.896844 (+ 18us) server_negotiation.cc:407] Connection header received
0620 18:45:03.896871 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:03.896872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:03.896883 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:03.897007 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:03.897345 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.897903 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.898669 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:03.898885 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:03.898953 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:03.901510 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:53509: BlockingRecv error: recv got EOF from 127.15.77.65:53509 (error 108)
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":229,"threads_started":1}
Jun 20 18:45:04 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:04 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:04.027060 24892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:04.022165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:56353)
0620 18:45:04.022313 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:04.022324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:04.022402 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:04.022584 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:04.022585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:04.022828 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:04.022887 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:04.022889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.023642 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.023644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:04.024186 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:04.024188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.024598 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.024599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:04.024660 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:04.025134 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:04.025138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:04.026999 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
W20260620 18:45:04.027204 24891 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:04.022254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:56353 (local address 127.15.77.126:38719)
0620 18:45:04.022326 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:04.022328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:04.022405 (+ 77us) server_negotiation.cc:407] Connection header received
0620 18:45:04.022456 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:04.022458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:04.022471 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:04.022561 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:04.022975 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.023558 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.024291 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.024496 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.024586 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:04.027120 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:56353: BlockingRecv error: recv got EOF from 127.15.77.66:56353 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 20 18:45:04 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:04 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:04.168962 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:04.163851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:47575)
0620 18:45:04.164014 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:04.164033 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:04.164126 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:04.164322 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:04.164323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:04.164504 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:04.164552 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:04.164554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.165332 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.165333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:04.165877 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:04.165879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.166259 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.166260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:04.166327 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:04.166824 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:04.166829 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:04.168895 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
W20260620 18:45:04.169117 24891 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:04.164023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:47575 (local address 127.15.77.126:38719)
0620 18:45:04.164103 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:04.164106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:04.164118 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:45:04.164218 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:04.164219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:04.164239 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:04.164301 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:04.164640 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.165238 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.165973 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.166174 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.166248 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:04.169028 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:47575: BlockingRecv error: recv got EOF from 127.15.77.67:47575 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 20 18:45:04 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:04 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:04.907114 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:04.902060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:56027)
0620 18:45:04.902248 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:04.902259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:04.902320 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:04.902454 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:04.902455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:04.902614 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:04.902680 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:04.902682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.903491 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.903492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:04.904043 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:04.904044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.904559 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.904560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:04.904647 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:04.905114 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:04.905119 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:04.907053 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":105,"threads_started":1}
W20260620 18:45:04.907254 24895 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:04.902165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:56027 (local address 127.15.77.126:38719)
0620 18:45:04.902320 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:04.902322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:04.902332 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:45:04.902360 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:04.902361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:04.902371 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:04.902433 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:04.902748 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.903259 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.904158 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:04.904477 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:04.904550 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:04.907168 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:56027: BlockingRecv error: recv got EOF from 127.15.77.65:56027 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
Jun 20 18:45:05 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:05 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:05.032130 24896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:05.027568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:60991)
0620 18:45:05.027713 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:05.027724 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:05.027799 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:05.027926 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:05.027926 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:05.028089 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:05.028139 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:05.028140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.028794 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.028795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:05.029310 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:05.029312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.029643 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.029644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:05.029696 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:05.030153 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:05.030158 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:05.032028 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
W20260620 18:45:05.032199 24895 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:05.027628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:60991 (local address 127.15.77.126:38719)
0620 18:45:05.027691 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:05.027693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:05.027780 (+ 87us) server_negotiation.cc:407] Connection header received
0620 18:45:05.027846 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:05.027847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:05.027858 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:05.027907 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:05.028208 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.028712 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.029387 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.029577 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.029631 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:05.032117 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:60991: BlockingRecv error: recv got EOF from 127.15.77.66:60991 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 20 18:45:05 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:05 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:05.174244 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:05.169528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:60119)
0620 18:45:05.169706 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:05.169717 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:05.169773 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:05.169913 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:05.169914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:05.170081 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:05.170131 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:05.170133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.170806 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.170807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:05.171397 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:05.171399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.171772 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.171773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:05.171829 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:05.172297 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:05.172302 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:05.174183 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":112,"threads_started":1}
W20260620 18:45:05.174391 24895 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:05.169601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:60119 (local address 127.15.77.126:38719)
0620 18:45:05.169647 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:05.169650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:05.169761 (+ 111us) server_negotiation.cc:407] Connection header received
0620 18:45:05.169818 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:05.169818 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:05.169832 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:05.169886 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:05.170200 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.170701 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.171474 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.171679 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.171735 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:05.174291 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:60119: BlockingRecv error: recv got EOF from 127.15.77.67:60119 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 20 18:45:05 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:05 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:05.912002 24898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:05.907646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:50753)
0620 18:45:05.907778 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:05.907794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:05.907845 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:05.907984 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:05.907985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:05.908178 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:05.908230 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:05.908232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.908877 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.908878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:05.909409 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:05.909411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.909745 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.909745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:05.909789 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:05.910219 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:05.910223 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:05.911940 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
W20260620 18:45:05.912134 24899 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:05.907718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:50753 (local address 127.15.77.126:38719)
0620 18:45:05.907848 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:05.907850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:05.907883 (+ 33us) server_negotiation.cc:407] Connection header received
0620 18:45:05.907905 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:05.907906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:05.907916 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:05.907967 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:05.908299 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.908807 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.909480 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:05.909679 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:05.909729 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:05.912043 (+ 2314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:50753: BlockingRecv error: recv got EOF from 127.15.77.65:50753 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
Jun 20 18:45:06 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:06 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:06.041678 24900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:06.032645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:49795)
0620 18:45:06.032915 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:06.032927 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:06.033019 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:06.033215 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:06.033216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:06.033385 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:06.033432 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:06.033433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.034110 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.034112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:06.034759 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:06.034761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.035255 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.035256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:06.035313 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:06.039556 (+ 4243us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:06.039561 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:06.041571 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":189,"threads_started":1}
W20260620 18:45:06.041738 24899 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:06.032836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:49795 (local address 127.15.77.126:38719)
0620 18:45:06.032913 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:06.032916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:06.032999 (+ 83us) server_negotiation.cc:407] Connection header received
0620 18:45:06.033099 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:06.033100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:06.033112 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:06.033206 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:06.033520 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.034023 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.034851 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.035179 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.035258 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:06.041673 (+ 6415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:49795: BlockingRecv error: recv got EOF from 127.15.77.66:49795 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 20 18:45:06 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:06 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:06.179941 24901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:06.174939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:53501)
0620 18:45:06.175124 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:06.175136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:06.175204 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:06.175508 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:06.175509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:06.175671 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:06.175721 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:06.175722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.176365 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.176366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:06.176909 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:06.176911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.177233 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.177234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:06.177279 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:06.177732 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:06.177737 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:06.179881 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":115,"threads_started":1}
W20260620 18:45:06.180052 24899 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:06.175016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:53501 (local address 127.15.77.126:38719)
0620 18:45:06.175101 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:06.175104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:06.175225 (+ 121us) server_negotiation.cc:407] Connection header received
0620 18:45:06.175258 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:06.175259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:06.175276 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:06.175399 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:06.175800 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.176294 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.176985 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.177161 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.177214 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:06.179976 (+ 2762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:53501: BlockingRecv error: recv got EOF from 127.15.77.67:53501 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 20 18:45:06 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:06 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:06.917429 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:06.912585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:37645)
0620 18:45:06.912785 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:06.912796 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:06.912865 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:06.913019 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:06.913020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:06.913180 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:06.913254 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:06.913256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.914209 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.914210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:06.914748 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:06.914750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.915085 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.915086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:06.915141 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:06.915595 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:06.915600 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:06.917369 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
W20260620 18:45:06.917546 24903 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:06.912648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:37645 (local address 127.15.77.126:38719)
0620 18:45:06.912816 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:06.912818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:06.912847 (+ 29us) server_negotiation.cc:407] Connection header received
0620 18:45:06.912923 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:06.912924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:06.912934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:06.912995 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:06.913328 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.914134 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.914820 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:06.915007 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:06.915056 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:06.917465 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:37645: BlockingRecv error: recv got EOF from 127.15.77.65:37645 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
Jun 20 18:45:07 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:07 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:07.047271 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:07.042280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:59165)
0620 18:45:07.042490 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:07.042505 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:07.042588 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:07.042829 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:07.042830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:07.042994 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:07.043045 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:07.043046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.043722 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.043723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:07.044269 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:07.044271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.044823 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.044824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:07.044906 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:07.045377 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:07.045381 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:07.047207 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":95,"threads_started":1}
W20260620 18:45:07.047441 24903 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:07.042466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:59165 (local address 127.15.77.126:38719)
0620 18:45:07.042555 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:07.042557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:07.042566 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:45:07.042711 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:07.042713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:07.042726 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:07.042811 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:07.043111 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.043638 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.044375 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.044744 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.044809 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:07.047371 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:59165: BlockingRecv error: recv got EOF from 127.15.77.66:59165 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 20 18:45:07 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:07 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:07.185442 24905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:07.180577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:59785)
0620 18:45:07.180746 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:07.180757 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:07.180818 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:07.180956 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:07.180957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:07.181121 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:07.181172 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:07.181173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.181844 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.181845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:07.182394 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:07.182396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.182721 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.182722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:07.182837 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:07.183289 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:07.183294 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:07.185376 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
W20260620 18:45:07.185592 24903 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:07.180694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:59785 (local address 127.15.77.126:38719)
0620 18:45:07.180751 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:07.180753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:07.180804 (+ 51us) server_negotiation.cc:407] Connection header received
0620 18:45:07.180863 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:07.180864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:07.180877 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:07.180932 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:07.181235 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.181742 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.182474 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.182653 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.182705 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:07.185494 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:59785: BlockingRecv error: recv got EOF from 127.15.77.67:59785 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 20 18:45:07 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:07 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:07.922827 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:07.917966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:55419)
0620 18:45:07.918132 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:07.918141 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:07.918212 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:07.918360 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:07.918361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:07.918522 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:07.918574 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:07.918576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.919213 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.919214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:07.919828 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:07.919831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.920409 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.920410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:07.920456 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:07.920998 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:07.921004 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:07.922765 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
W20260620 18:45:07.922992 24907 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:07.918062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:55419 (local address 127.15.77.126:38719)
0620 18:45:07.918168 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:07.918169 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:07.918199 (+ 30us) server_negotiation.cc:407] Connection header received
0620 18:45:07.918271 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:07.918272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:07.918281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:07.918348 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:07.918644 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.919141 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.919923 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:07.920322 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:07.920396 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:07.922887 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:55419: BlockingRecv error: recv got EOF from 127.15.77.65:55419 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":48,"threads_started":1}
Jun 20 18:45:08 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:08 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:08.052562 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:08.047845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:52947)
0620 18:45:08.048003 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:08.048015 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:08.048085 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:08.048254 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:08.048255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:08.048439 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:08.048487 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:08.048488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.049171 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.049172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:08.049736 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:08.049738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.050115 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.050116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:08.050176 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:08.050690 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:08.050695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:08.052502 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
W20260620 18:45:08.052711 24907 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:08.047883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:52947 (local address 127.15.77.126:38719)
0620 18:45:08.047956 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:08.047958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:08.048124 (+ 166us) server_negotiation.cc:407] Connection header received
0620 18:45:08.048148 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:08.048149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:08.048158 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:08.048245 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:08.048569 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.049073 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.049831 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.050027 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.050116 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:08.052632 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:52947: BlockingRecv error: recv got EOF from 127.15.77.66:52947 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 18:45:08 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:08 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:08.191382 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:08.186108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:57007)
0620 18:45:08.186296 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:08.186310 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:08.186393 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:08.186614 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:08.186615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:08.186793 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:08.186845 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:08.186846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.187689 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.187690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:08.188227 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:08.188229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.188604 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.188605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:08.188645 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:08.189126 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:08.189131 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:08.191253 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":80,"threads_started":1}
W20260620 18:45:08.191432 24907 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:08.186250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:57007 (local address 127.15.77.126:38719)
0620 18:45:08.186331 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:08.186333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:08.186447 (+ 114us) server_negotiation.cc:407] Connection header received
0620 18:45:08.186484 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:08.186485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:08.186500 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:08.186768 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:08.186946 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.187615 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.188309 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.188534 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.188591 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:08.191365 (+ 2774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:57007: BlockingRecv error: recv got EOF from 127.15.77.67:57007 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 20 18:45:08 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:08 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:08.928282 24910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:08.923498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:47647)
0620 18:45:08.923639 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:08.923655 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:08.923734 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:08.923940 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:08.923942 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:08.924159 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:08.924222 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:08.924224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.924929 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.924931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:08.925463 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:08.925465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.925794 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.925795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:08.925836 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:08.926287 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:08.926292 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:08.928170 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
W20260620 18:45:08.928334 24911 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:08.923609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:47647 (local address 127.15.77.126:38719)
0620 18:45:08.923732 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:08.923736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:08.923755 (+ 19us) server_negotiation.cc:407] Connection header received
0620 18:45:08.923833 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:08.923834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:08.923846 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:08.923917 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:08.924311 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.924851 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.925539 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:08.925732 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:08.925783 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:08.928259 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:47647: BlockingRecv error: recv got EOF from 127.15.77.65:47647 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
Jun 20 18:45:09 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:09 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:09.062057 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:09.053095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:40879)
0620 18:45:09.053295 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:09.053305 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:09.053364 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:09.053504 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:09.053505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:09.053675 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:09.053727 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:09.053729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.054352 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.054353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:09.055120 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:09.055124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.055558 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.055559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:09.055601 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:09.060133 (+ 4532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:09.060138 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:09.061946 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":109,"threads_started":1}
W20260620 18:45:09.062112 24911 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:09.053190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:40879 (local address 127.15.77.126:38719)
0620 18:45:09.053255 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:09.053258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:09.053361 (+ 103us) server_negotiation.cc:407] Connection header received
0620 18:45:09.053412 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:09.053413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:09.053424 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:09.053484 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:09.053789 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.054282 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.055210 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.055495 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.055549 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:09.062045 (+ 6496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:40879: BlockingRecv error: recv got EOF from 127.15.77.66:40879 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 20 18:45:09 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:09 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:09.197192 24913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:09.191911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:52707)
0620 18:45:09.192049 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:09.192060 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:09.192167 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:09.192347 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:09.192349 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:09.192549 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:09.192617 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:09.192619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.193335 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.193337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:09.193881 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:09.193883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.194284 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.194285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:09.194344 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:09.194872 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:09.194879 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:09.197097 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
W20260620 18:45:09.197270 24911 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:09.191951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:52707 (local address 127.15.77.126:38719)
0620 18:45:09.191999 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:09.192002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:09.192147 (+ 145us) server_negotiation.cc:407] Connection header received
0620 18:45:09.192256 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:09.192257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:09.192272 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:09.192325 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:09.192709 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.193230 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.193972 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.194189 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.194280 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:09.197203 (+ 2923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:52707: BlockingRecv error: recv got EOF from 127.15.77.67:52707 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 20 18:45:09 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:09 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:09.933995 24914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:09.928831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:34417)
0620 18:45:09.928990 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:09.929002 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:09.929063 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:09.929322 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:09.929323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:09.929504 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:09.929560 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:09.929562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.930316 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.930317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:09.930871 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:09.930873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.931361 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.931362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:09.931420 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:09.931895 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:09.931900 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:09.933891 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
W20260620 18:45:09.934052 24915 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:09.929009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:34417 (local address 127.15.77.126:38719)
0620 18:45:09.929145 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:09.929147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:09.929156 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:45:09.929180 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:09.929181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:09.929191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:09.929251 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:09.929640 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.930240 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.930953 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:09.931223 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:09.931312 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:09.933978 (+ 2666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:34417: BlockingRecv error: recv got EOF from 127.15.77.65:34417 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
Jun 20 18:45:10 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:10 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:10.067657 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:10.062684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:37685)
0620 18:45:10.062836 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:10.062848 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:10.062911 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:10.063111 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:10.063112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:10.063288 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:10.063412 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:10.063414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.064396 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.064398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:10.064963 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:10.064965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.065299 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.065300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:10.065340 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:10.065803 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:10.065807 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:10.067596 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
W20260620 18:45:10.067816 24915 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:10.062884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:37685 (local address 127.15.77.126:38719)
0620 18:45:10.062970 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:10.062972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:10.062981 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:45:10.063012 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:10.063013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:10.063023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:10.063090 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:10.063498 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.064306 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.065038 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.065227 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.065280 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:10.067729 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:37685: BlockingRecv error: recv got EOF from 127.15.77.66:37685 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 20 18:45:10 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:10 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:10.202189 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:10.197952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:38533)
0620 18:45:10.198088 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:10.198099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:10.198157 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:10.198284 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:10.198285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:10.198443 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:10.198492 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:10.198494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.199108 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.199108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:10.199657 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:10.199659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.199996 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.199997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:10.200038 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:10.200470 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:10.200475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:10.202121 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
W20260620 18:45:10.202281 24915 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:10.198017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:38533 (local address 127.15.77.126:38719)
0620 18:45:10.198103 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:10.198106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:10.198147 (+ 41us) server_negotiation.cc:407] Connection header received
0620 18:45:10.198204 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:10.198204 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:10.198217 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:10.198264 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:10.198557 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.199045 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.199747 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.199930 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.199985 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:10.202212 (+ 2227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:38533: BlockingRecv error: recv got EOF from 127.15.77.67:38533 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 20 18:45:10 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:10 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:10.940507 24918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:10.934677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:42179)
0620 18:45:10.934832 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:10.934844 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:10.934920 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:10.935366 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:10.935368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:10.935609 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:10.935661 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:10.935663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.936504 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.936505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:10.937248 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:10.937251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.937641 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.937642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:10.937706 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:10.938240 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:10.938246 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:10.940433 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
W20260620 18:45:10.940671 24919 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:10.934953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:42179 (local address 127.15.77.126:38719)
0620 18:45:10.935101 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:10.935103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:10.935112 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:45:10.935138 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:10.935139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:10.935149 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:10.935303 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:10.935761 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.936415 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.937334 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:10.937554 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:10.937639 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:10.940569 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:42179: BlockingRecv error: recv got EOF from 127.15.77.65:42179 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
Jun 20 18:45:11 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:11 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:11.084903 24920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:11.068108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:52535)
0620 18:45:11.068263 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:11.068280 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:11.068337 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:11.068440 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:11.068441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:11.068640 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:11.068689 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:11.068691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.069328 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.069328 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:11.069864 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:11.069866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.070184 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.070185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:11.070225 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:11.082829 (+ 12604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:11.082835 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:11.084828 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
W20260620 18:45:11.085074 24919 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:11.068139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:52535 (local address 127.15.77.126:38719)
0620 18:45:11.068186 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:11.068188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:11.068349 (+ 161us) server_negotiation.cc:407] Connection header received
0620 18:45:11.068377 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:11.068378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:11.068389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:11.068435 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:11.068754 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.069259 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.069939 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.070119 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.070169 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:11.084991 (+ 14822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:52535: BlockingRecv error: recv got EOF from 127.15.77.66:52535 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 20 18:45:11 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:11 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:11.207758 24921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:11.202724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:56699)
0620 18:45:11.202902 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:11.202916 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:11.203016 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:11.203181 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:11.203182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:11.203384 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:11.203433 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:11.203435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.204126 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.204128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:11.204653 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:11.204655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.205044 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.205045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:11.205111 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:11.205623 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:11.205628 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:11.207682 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
W20260620 18:45:11.207894 24919 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:11.202828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:56699 (local address 127.15.77.126:38719)
0620 18:45:11.202917 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:11.202919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:11.202997 (+ 78us) server_negotiation.cc:407] Connection header received
0620 18:45:11.203100 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:11.203101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:11.203114 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:11.203161 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:11.203520 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.204029 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.204747 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.204956 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.205049 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:11.207784 (+ 2735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:56699: BlockingRecv error: recv got EOF from 127.15.77.67:56699 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 20 18:45:11 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:11 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:11.945961 24922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:11.941062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:46299)
0620 18:45:11.941280 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:11.941292 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:11.941363 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:11.941502 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:11.941503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:11.941682 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:11.941732 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:11.941734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.942384 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.942384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:11.942949 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:11.942951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.943345 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.943346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:11.943389 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:11.943857 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:11.943862 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:11.945870 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":108,"threads_started":1}
W20260620 18:45:11.946081 24923 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:11.941168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:46299 (local address 127.15.77.126:38719)
0620 18:45:11.941283 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:11.941286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:11.941349 (+ 63us) server_negotiation.cc:407] Connection header received
0620 18:45:11.941410 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:11.941411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:11.941421 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:11.941480 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:11.941803 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.942313 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.943034 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:11.943246 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:11.943303 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:11.945987 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:46299: BlockingRecv error: recv got EOF from 127.15.77.65:46299 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
Jun 20 18:45:12 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:12 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:12.090312 24924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:12.085430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:58797)
0620 18:45:12.085594 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:12.085605 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:12.085677 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:12.085841 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:12.085842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:12.086009 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:12.086060 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:12.086061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.086740 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.086741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:12.087277 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:12.087279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.087798 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.087799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:12.087870 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:12.088399 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:12.088405 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:12.090219 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
W20260620 18:45:12.090396 24923 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:12.085501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:58797 (local address 127.15.77.126:38719)
0620 18:45:12.085598 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:12.085600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:12.085671 (+ 71us) server_negotiation.cc:407] Connection header received
0620 18:45:12.085756 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:12.085757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:12.085769 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:12.085813 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:12.086131 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.086624 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.087448 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.087708 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.087787 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:12.090323 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:58797: BlockingRecv error: recv got EOF from 127.15.77.66:58797 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Jun 20 18:45:12 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:12 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:12.213151 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:12.208390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:49601)
0620 18:45:12.208557 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:12.208570 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:12.208624 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:12.208805 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:12.208806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:12.209055 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:12.209118 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:12.209121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.209766 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.209767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:12.210301 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:12.210303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.210671 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.210671 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:12.210711 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:12.211174 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:12.211179 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:12.213053 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
W20260620 18:45:12.213194 24923 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:12.208579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:49601 (local address 127.15.77.126:38719)
0620 18:45:12.208653 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:12.208656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:12.208665 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:45:12.208710 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:12.208711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:12.208723 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:12.208778 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:12.209204 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.209694 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.210381 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.210590 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.210649 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:12.213130 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:49601: BlockingRecv error: recv got EOF from 127.15.77.67:49601 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 20 18:45:12 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:12 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:12.951431 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:12.946446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:33757)
0620 18:45:12.946626 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:12.946637 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:12.946694 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:12.946942 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:12.946943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:12.947125 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:12.947191 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:12.947193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.947953 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.947954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:12.948529 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:12.948531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.948930 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.948931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:12.948973 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:12.949461 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:12.949466 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:12.951299 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
W20260620 18:45:12.951532 24927 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:12.946598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:33757 (local address 127.15.77.126:38719)
0620 18:45:12.946781 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:12.946783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:12.946790 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:45:12.946812 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:12.946813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:12.946823 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:12.946887 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:12.947268 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.947878 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.948620 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:12.948854 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:12.948916 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:12.951429 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:33757: BlockingRecv error: recv got EOF from 127.15.77.65:33757 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":93,"threads_started":1}
Jun 20 18:45:13 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:13 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:13.096438 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:13.090991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:51853)
0620 18:45:13.091235 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:13.091248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:13.091391 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:13.091642 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:13.091644 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:13.091870 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:13.091920 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:13.091921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.092701 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.092702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:13.093322 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:13.093324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.093765 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.093766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:13.093827 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:13.094338 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:13.094343 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:13.096317 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":105,"threads_started":1}
W20260620 18:45:13.096493 24927 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:13.091111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:51853 (local address 127.15.77.126:38719)
0620 18:45:13.091221 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:13.091224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:13.091348 (+ 124us) server_negotiation.cc:407] Connection header received
0620 18:45:13.091540 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:13.091541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:13.091554 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:13.091633 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:13.092034 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.092602 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.093428 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.093663 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.093771 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:13.096411 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:51853: BlockingRecv error: recv got EOF from 127.15.77.66:51853 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 20 18:45:13 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:13 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:13.218271 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:13.213741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:33823)
0620 18:45:13.213872 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:13.213884 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:13.213932 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:13.214077 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:13.214078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:13.214243 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:13.214290 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:13.214291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.214968 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.214969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:13.215591 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:13.215594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.215922 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.215922 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:13.215967 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:13.216435 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:13.216439 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:13.218198 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
W20260620 18:45:13.218408 24927 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:13.213833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:33823 (local address 127.15.77.126:38719)
0620 18:45:13.213931 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:13.213934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:13.213950 (+ 16us) server_negotiation.cc:407] Connection header received
0620 18:45:13.213981 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:13.213982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:13.213996 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:13.214060 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:13.214377 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.214893 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.215679 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.215857 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.215908 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:13.218299 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:33823: BlockingRecv error: recv got EOF from 127.15.77.67:33823 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 20 18:45:13 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:13 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:13.956907 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:13.951996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:50497)
0620 18:45:13.952209 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:13.952223 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:13.952296 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:13.952479 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:13.952480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:13.952677 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:13.952729 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:13.952731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.953428 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.953429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:13.954010 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:13.954012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.954361 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.954362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:13.954403 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:13.954979 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:13.954985 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:13.956837 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":91,"threads_started":1}
W20260620 18:45:13.957039 24931 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:13.952143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:50497 (local address 127.15.77.126:38719)
0620 18:45:13.952303 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:13.952305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:13.952317 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:45:13.952381 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:13.952381 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:13.952392 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:13.952455 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:13.952803 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.953327 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.954094 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:13.954293 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:13.954344 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:13.956951 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:50497: BlockingRecv error: recv got EOF from 127.15.77.65:50497 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
Jun 20 18:45:14 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:14 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:14.101670 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:14.096906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:42591)
0620 18:45:14.097043 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:14.097061 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:14.097117 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:14.097245 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:14.097246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:14.097404 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:14.097453 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:14.097455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.098153 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.098154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:14.098690 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:14.098692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.099040 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.099041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:14.099093 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:14.099591 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:14.099596 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:14.101609 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
W20260620 18:45:14.101861 24931 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:14.096987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:42591 (local address 127.15.77.126:38719)
0620 18:45:14.097046 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:14.097048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:14.097118 (+ 70us) server_negotiation.cc:407] Connection header received
0620 18:45:14.097165 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:14.097166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:14.097177 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:14.097230 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:14.097510 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.098083 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.098767 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.098952 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.099004 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:14.101783 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:42591: BlockingRecv error: recv got EOF from 127.15.77.66:42591 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 20 18:45:14 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:14 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:14.223652 24933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:14.218771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:43629)
0620 18:45:14.218929 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:14.218939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:14.218996 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:14.219130 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:14.219131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:14.219299 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:14.219411 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:14.219413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.220063 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.220064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:14.220774 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:14.220778 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.221242 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.221243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:14.221307 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:14.221748 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:14.221752 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:14.223592 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
W20260620 18:45:14.223745 24931 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:14.218863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:43629 (local address 127.15.77.126:38719)
0620 18:45:14.218925 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:14.218927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:14.218981 (+ 54us) server_negotiation.cc:407] Connection header received
0620 18:45:14.219053 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:14.219054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:14.219065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:14.219107 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:14.219489 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.219994 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.220865 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.221155 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.221240 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:14.223695 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:43629: BlockingRecv error: recv got EOF from 127.15.77.67:43629 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 20 18:45:14 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:14 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:14.962189 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:14.957494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:54701)
0620 18:45:14.957644 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:14.957655 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:14.957710 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:14.957863 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:14.957864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:14.958028 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:14.958078 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:14.958079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.958754 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.958755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:14.959286 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:14.959288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.959730 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.959731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:14.959778 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:14.960218 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:14.960222 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:14.962073 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
W20260620 18:45:14.962298 24935 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:14.957591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:54701 (local address 127.15.77.126:38719)
0620 18:45:14.957697 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:14.957699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:14.957709 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:45:14.957787 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:14.957788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:14.957798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:14.957852 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:14.958160 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.958675 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.959466 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:14.959663 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:14.959716 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:14.962208 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:54701: BlockingRecv error: recv got EOF from 127.15.77.65:54701 (error 108)
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":44,"threads_started":1}
Jun 20 18:45:15 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:15 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:15.106763 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:15.102163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:39683)
0620 18:45:15.102327 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:15.102337 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:15.102393 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:15.102540 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:15.102541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:15.102731 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:15.102780 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:15.102782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.103458 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.103459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:15.103989 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:15.103991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.104323 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.104324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:15.104363 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:15.104927 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:15.104932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:15.106701 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
W20260620 18:45:15.106868 24935 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:15.102274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:39683 (local address 127.15.77.126:38719)
0620 18:45:15.102337 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:15.102339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:15.102378 (+ 39us) server_negotiation.cc:407] Connection header received
0620 18:45:15.102449 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:15.102450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:15.102460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:15.102528 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:15.102849 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.103393 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.104061 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.104254 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.104311 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:15.106796 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:39683: BlockingRecv error: recv got EOF from 127.15.77.66:39683 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Jun 20 18:45:15 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:15 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:15.229645 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:15.224200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:54901)
0620 18:45:15.224384 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:15.224396 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:15.224461 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:15.224631 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:15.224633 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:15.224812 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:15.224872 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:15.224874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.225908 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.225909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:15.226797 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:15.226802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.227364 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.227366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:15.227427 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:15.227871 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:15.227876 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:15.229570 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
W20260620 18:45:15.229760 24935 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:15.224394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:54901 (local address 127.15.77.126:38719)
0620 18:45:15.224463 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:15.224465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:15.224475 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:45:15.224527 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:15.224528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:15.224542 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:15.224622 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:15.224954 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.225775 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.226887 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.227230 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.227335 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:15.229661 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:54901: BlockingRecv error: recv got EOF from 127.15.77.67:54901 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 18:45:15 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:15 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:15.967396 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:15.962722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:40359)
0620 18:45:15.962869 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:15.962883 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:15.962946 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:15.963120 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:15.963121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:15.963409 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:15.963485 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:15.963488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.964208 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.964209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:15.964850 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:15.964852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.965197 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.965197 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:15.965239 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:15.965661 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:15.965665 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:15.967336 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
W20260620 18:45:15.967522 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:15.962788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:40359 (local address 127.15.77.126:38719)
0620 18:45:15.962940 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:15.962942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:15.962950 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:45:15.963017 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:15.963018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:15.963028 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:15.963085 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:15.963563 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.964127 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.964927 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:15.965126 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:15.965194 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:15.967440 (+ 2246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:40359: BlockingRecv error: recv got EOF from 127.15.77.65:40359 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
Jun 20 18:45:16 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:16 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:16.115077 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:16.107289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:49221)
0620 18:45:16.107441 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:16.107455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:16.107521 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:16.107683 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:16.107684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:16.107868 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:16.107914 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:16.107915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.108687 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.108688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:16.109240 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:16.109242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.109642 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.109643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:16.109702 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:16.113158 (+ 3456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:16.113163 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:16.114981 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":127,"threads_started":1}
W20260620 18:45:16.115181 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:16.107411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:49221 (local address 127.15.77.126:38719)
0620 18:45:16.107512 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:16.107514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:16.107527 (+ 13us) server_negotiation.cc:407] Connection header received
0620 18:45:16.107577 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:16.107578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:16.107591 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:16.107671 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:16.107999 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.108570 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.109339 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.109551 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.109642 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:16.115116 (+ 5474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:49221: BlockingRecv error: recv got EOF from 127.15.77.66:49221 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 20 18:45:16 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:16 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.67@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:16.235384 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:16.230230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.67:49589)
0620 18:45:16.230368 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:16.230381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:16.230454 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:16.230588 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:16.230589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:16.230761 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:16.230811 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:16.230813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.231702 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.231704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:16.232237 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:16.232238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.232606 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.232607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:16.232665 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:16.233267 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:16.233271 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:16.235267 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
W20260620 18:45:16.235447 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:16.230314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:49589 (local address 127.15.77.126:38719)
0620 18:45:16.230397 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:16.230399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:16.230438 (+ 39us) server_negotiation.cc:407] Connection header received
0620 18:45:16.230503 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:16.230504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:16.230518 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:16.230574 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:16.230906 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.231618 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.232327 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.232533 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.232599 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:16.235397 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.67:49589: BlockingRecv error: recv got EOF from 127.15.77.67:49589 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 18:45:16 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:16 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.65@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:16.972949 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:16.967979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.65:33537)
0620 18:45:16.968171 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:16.968184 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:16.968243 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:16.968403 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:16.968404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:16.968581 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:16.968634 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:16.968636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.969294 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.969294 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:16.969844 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:16.969846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.970252 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.970253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:16.970330 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:16.970878 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:16.970884 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:16.972878 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":98,"threads_started":1}
W20260620 18:45:16.973124 24943 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:16.968089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:33537 (local address 127.15.77.126:38719)
0620 18:45:16.968213 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:16.968215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:16.968229 (+ 14us) server_negotiation.cc:407] Connection header received
0620 18:45:16.968306 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:16.968307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:16.968317 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:16.968378 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:16.968703 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.969218 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.969948 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:16.970174 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:16.970240 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:16.973034 (+ 2794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.65:33537: BlockingRecv error: recv got EOF from 127.15.77.65:33537 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":53,"threads_started":1}
Jun 20 18:45:17 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
Jun 20 18:45:17 dist-test-slave-tvl9 krb5kdc[23885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.77.66@KRBTEST.COM for oryx/127.15.77.126@KRBTEST.COM, Server not found in Kerberos database
I20260620 18:45:17.121140 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:17.115833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:38719 (local address 127.15.77.66:44263)
0620 18:45:17.115984 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:17.115997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:17.116071 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:17.116282 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:17.116283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:17.116496 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:17.116561 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:17.116563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.117306 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.117307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:17.117941 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:17.117944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.118417 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.118418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:17.118469 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:17.118957 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:17.118963 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:17.121023 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.77.126:38719: Server oryx/127.15.77.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
W20260620 18:45:17.121209 24943 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:17.115895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:44263 (local address 127.15.77.126:38719)
0620 18:45:17.115956 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:17.115959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:17.116094 (+ 135us) server_negotiation.cc:407] Connection header received
0620 18:45:17.116161 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:17.116162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:17.116178 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:17.116260 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:17.116666 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.117212 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.118039 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.118344 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.118414 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:17.121159 (+ 2745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.77.66:44263: BlockingRecv error: recv got EOF from 127.15.77.66:44263 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:45:17.150559 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 24468
I20260620 18:45:17.155443 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 24620
I20260620 18:45:17.159855 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 24752
I20260620 18:45:17.164057 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 24398
2026-06-20T18:45:17Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17491 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-20T18:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:45:17Z Disabled control of system clock
I20260620 18:45:17.192605 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:39437
--webserver_interface=127.15.77.126
--webserver_port=0
--builtin_ntp_servers=127.15.77.84:44529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:39437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 18:45:17.279031 24948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:17.279266 24948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:17.279287 24948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:17.280609 24948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:45:17.280658 24948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:17.280670 24948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:17.280683 24948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:45:17.280696 24948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:45:17.282135 24948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:44529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:39437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:39437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:17.282425 24948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:17.282815 24948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:17.288122 24956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:17.288141 24948 server_base.cc:1034] running on GCE node
W20260620 18:45:17.288110 24953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:17.288110 24954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:17.288650 24948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:17.288955 24948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:17.290107 24948 hybrid_clock.cc:648] HybridClock initialized: now 1781981117290079 us; error 51 us; skew 500 ppm
I20260620 18:45:17.291481 24948 webserver.cc:460] Webserver started at http://127.15.77.126:36423/ using document root <none> and password file <none>
I20260620 18:45:17.291766 24948 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:17.291812 24948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:17.291991 24948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:17.293159 24948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "3b108e6427cc4a2c858dcbeec20a6d2b"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "8daa2a208c681d215d4f67d94beff814"
server_key_iv: "2130caf32b43b9d3000a8f90a56f1c05"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.293542 24948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "3b108e6427cc4a2c858dcbeec20a6d2b"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "8daa2a208c681d215d4f67d94beff814"
server_key_iv: "2130caf32b43b9d3000a8f90a56f1c05"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.294898 24948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:45:17.295773 24962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.295996 24948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 18:45:17.296109 24948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "3b108e6427cc4a2c858dcbeec20a6d2b"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "8daa2a208c681d215d4f67d94beff814"
server_key_iv: "2130caf32b43b9d3000a8f90a56f1c05"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.296190 24948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:17.308812 24948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:17.309134 24948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:17.309245 24948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:17.313195 24948 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:39437
I20260620 18:45:17.313278 25014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:39437 every 8 connection(s)
I20260620 18:45:17.313609 24948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:45:17.314348 25015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:45:17.317101 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 24948
I20260620 18:45:17.317114 25015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b: Bootstrap starting.
I20260620 18:45:17.317185 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:45:17.317399 15669 external_mini_cluster.cc:1428] Setting key a780000aa642370b77654df361c5d23e
I20260620 18:45:17.317883 25015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b: Neither blocks nor log segments found. Creating new log.
I20260620 18:45:17.318316 25015 log.cc:824] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b: Log is configured to *not* fsync() on all Append() calls
I20260620 18:45:17.319075 25015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b: No bootstrap required, opened a new log
I20260620 18:45:17.321019 25015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b108e6427cc4a2c858dcbeec20a6d2b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39437 } }
I20260620 18:45:17.321162 25015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:45:17.321187 25015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b108e6427cc4a2c858dcbeec20a6d2b, State: Initialized, Role: FOLLOWER
I20260620 18:45:17.321316 25015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b108e6427cc4a2c858dcbeec20a6d2b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39437 } }
I20260620 18:45:17.321604 25015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:45:17.321653 25015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:45:17.321734 25015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:45:17.322286 25015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b108e6427cc4a2c858dcbeec20a6d2b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39437 } }
I20260620 18:45:17.322409 25015 leader_election.cc:302] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b108e6427cc4a2c858dcbeec20a6d2b; no voters:
I20260620 18:45:17.322574 25015 leader_election.cc:288] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:45:17.322635 25020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:45:17.322834 25020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [term 1 LEADER]: Becoming Leader. State: Replica: 3b108e6427cc4a2c858dcbeec20a6d2b, State: Running, Role: LEADER
I20260620 18:45:17.322906 25015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:45:17.322965 25020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b108e6427cc4a2c858dcbeec20a6d2b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39437 } }
I20260620 18:45:17.323354 25021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b108e6427cc4a2c858dcbeec20a6d2b. Latest consensus state: current_term: 1 leader_uuid: "3b108e6427cc4a2c858dcbeec20a6d2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b108e6427cc4a2c858dcbeec20a6d2b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39437 } } }
I20260620 18:45:17.323520 25021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [sys.catalog]: This master's current role is: LEADER
I20260620 18:45:17.323997 25029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:45:17.324038 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:17.318004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33746 (local address 127.15.77.126:39437)
0620 18:45:17.318244 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:17.318253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:17.318289 (+ 36us) server_negotiation.cc:407] Connection header received
0620 18:45:17.318842 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:17.318851 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:17.318929 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:17.319080 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:17.319345 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.320760 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.321702 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.322181 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.322321 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:17.323130 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:17.323134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:17.323135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:45:17.323379 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:17.323432 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:45:17.323457 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:17.323522 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:17.323731 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 18:45:17.323836 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260620 18:45:17.324045 25022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b108e6427cc4a2c858dcbeec20a6d2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b108e6427cc4a2c858dcbeec20a6d2b" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39437 } } }
I20260620 18:45:17.324149 25022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b [sys.catalog]: This master's current role is: LEADER
I20260620 18:45:17.324977 25029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:45:17.326467 25029 catalog_manager.cc:1347] Generated new cluster ID: 2b0cc24bf372442f8da5f739314a4f2d
I20260620 18:45:17.326546 25029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:45:17.341821 25029 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:45:17.342368 25029 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:45:17.353008 25029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b108e6427cc4a2c858dcbeec20a6d2b: Generated new TSK 0
I20260620 18:45:17.353255 25029 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:45:17.363123 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39437
--builtin_ntp_servers=127.15.77.84:44529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 18:45:17.445164 25039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:17.445415 25039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:17.445454 25039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:17.446645 25039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:17.446714 25039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:17.446758 25039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:45:17.448223 25039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:44529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:17.448498 25039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:17.448817 25039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:17.453989 25044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:17.453989 25047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:17.454056 25039 server_base.cc:1034] running on GCE node
W20260620 18:45:17.453994 25045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:17.454397 25039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:17.454663 25039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:17.455837 25039 hybrid_clock.cc:648] HybridClock initialized: now 1781981117455824 us; error 28 us; skew 500 ppm
I20260620 18:45:17.457259 25039 webserver.cc:460] Webserver started at http://127.15.77.65:45217/ using document root <none> and password file <none>
I20260620 18:45:17.457509 25039 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:17.457579 25039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:17.457742 25039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:17.458868 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "b887c947d6c54178abb7c20cab4d5834"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "329f7e42f84f5071953507d55a067c33"
server_key_iv: "3862c28394e6f8fbbbc250d362e62a4f"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.459236 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "b887c947d6c54178abb7c20cab4d5834"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "329f7e42f84f5071953507d55a067c33"
server_key_iv: "3862c28394e6f8fbbbc250d362e62a4f"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.460577 25039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:45:17.461292 25053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.461464 25039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:45:17.461562 25039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "b887c947d6c54178abb7c20cab4d5834"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "329f7e42f84f5071953507d55a067c33"
server_key_iv: "3862c28394e6f8fbbbc250d362e62a4f"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.461649 25039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:17.482113 25039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:17.482478 25039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:17.482645 25039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:17.482896 25039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:17.483254 25039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:17.483310 25039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.483421 25039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:17.483454 25039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.489334 25039 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:42153
I20260620 18:45:17.489416 25166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:42153 every 8 connection(s)
I20260620 18:45:17.489755 25039 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:45:17.494653 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:17.490495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:49945 (local address 127.15.77.126:39437)
0620 18:45:17.490576 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:17.490578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:17.490781 (+ 203us) server_negotiation.cc:407] Connection header received
0620 18:45:17.491261 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:17.491262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:17.491273 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:17.491385 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:17.492011 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.492499 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.493425 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.493601 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.493669 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:17.494410 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:17.494411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:17.494412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:45:17.494416 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:17.494429 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:45:17.494433 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:17.494476 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:17.494556 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 18:45:17.494578 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 18:45:17.494861 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:17.490457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:39437 (local address 127.15.77.65:49945)
0620 18:45:17.490654 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:17.490689 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:17.491175 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:17.491444 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:17.491448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:17.491456 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:17.491940 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:17.491945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.492576 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.492576 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:17.493355 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:17.493357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.493675 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.493676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:17.493780 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:17.494282 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:45:17.494282 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:17.494309 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:45:17.494332 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:17.494487 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:17.494491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:17.494493 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:45:17.494546 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 18:45:17.494623 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260620 18:45:17.495355 25167 heartbeater.cc:347] Connected to a master server at 127.15.77.126:39437
I20260620 18:45:17.495451 25167 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:17.495604 25167 heartbeater.cc:511] Master 127.15.77.126:39437 requested a full tablet report, sending...
I20260620 18:45:17.496062 24979 ts_manager.cc:194] Registered new tserver with Master: b887c947d6c54178abb7c20cab4d5834 (127.15.77.65:42153)
I20260620 18:45:17.496699 24979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.65:49945
I20260620 18:45:17.498445 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 25039
I20260620 18:45:17.498517 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:45:17.498756 15669 external_mini_cluster.cc:1428] Setting key 18b55468d2657a5bbf1f2dff702c5619
I20260620 18:45:17.499639 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39437
--builtin_ntp_servers=127.15.77.84:44529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 18:45:17.579615 25170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:17.579869 25170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:17.579905 25170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:17.581053 25170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:17.581122 25170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:17.581172 25170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:45:17.582518 25170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:44529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:17.582791 25170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:17.583091 25170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:17.588236 25176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:17.588240 25178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:17.588239 25175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:17.588526 25170 server_base.cc:1034] running on GCE node
I20260620 18:45:17.588775 25170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:17.589016 25170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:17.590180 25170 hybrid_clock.cc:648] HybridClock initialized: now 1781981117590170 us; error 27 us; skew 500 ppm
I20260620 18:45:17.591588 25170 webserver.cc:460] Webserver started at http://127.15.77.66:38435/ using document root <none> and password file <none>
I20260620 18:45:17.591848 25170 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:17.591918 25170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:17.592064 25170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:17.593170 25170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "a2dc58aca887433f82f12b4bb58ac872"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "70b421fcb966c8916a69a1b4baad44fa"
server_key_iv: "02c922b598777b20e17524e73440a6a1"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.593547 25170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "a2dc58aca887433f82f12b4bb58ac872"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "70b421fcb966c8916a69a1b4baad44fa"
server_key_iv: "02c922b598777b20e17524e73440a6a1"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.594878 25170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:45:17.595691 25184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.595868 25170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.595966 25170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "a2dc58aca887433f82f12b4bb58ac872"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "70b421fcb966c8916a69a1b4baad44fa"
server_key_iv: "02c922b598777b20e17524e73440a6a1"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.596050 25170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:17.607846 25170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:17.608196 25170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:17.608353 25170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:17.608592 25170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:17.608942 25170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:17.608996 25170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.609042 25170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:17.609067 25170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.615018 25170 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:41811
I20260620 18:45:17.615067 25297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:41811 every 8 connection(s)
I20260620 18:45:17.615495 25170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:45:17.620419 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:17.616240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:40857 (local address 127.15.77.126:39437)
0620 18:45:17.616286 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:17.616288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:17.616536 (+ 248us) server_negotiation.cc:407] Connection header received
0620 18:45:17.617034 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:17.617036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:17.617045 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:17.617098 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:17.617732 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.618212 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.619158 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.619402 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.619479 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:17.620209 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:17.620211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:17.620211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:45:17.620216 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:17.620229 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:45:17.620233 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:17.620275 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:17.620337 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 18:45:17.620358 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:45:17.620601 25300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:17.616241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:39437 (local address 127.15.77.66:40857)
0620 18:45:17.616409 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:17.616443 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:17.616950 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:17.617154 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:17.617158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:17.617166 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:17.617664 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:17.617668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.618289 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.618289 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:17.619083 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:17.619085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.619489 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.619490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:17.619624 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:17.620086 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:45:17.620086 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:17.620113 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:45:17.620134 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:17.620275 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:17.620277 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:17.620280 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:45:17.620326 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 18:45:17.620399 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":86,"threads_started":1}
I20260620 18:45:17.621119 25298 heartbeater.cc:347] Connected to a master server at 127.15.77.126:39437
I20260620 18:45:17.621218 25298 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:17.621392 25298 heartbeater.cc:511] Master 127.15.77.126:39437 requested a full tablet report, sending...
I20260620 18:45:17.621812 24979 ts_manager.cc:194] Registered new tserver with Master: a2dc58aca887433f82f12b4bb58ac872 (127.15.77.66:41811)
I20260620 18:45:17.622195 24979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.66:40857
I20260620 18:45:17.624219 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 25170
I20260620 18:45:17.624313 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:45:17.624505 15669 external_mini_cluster.cc:1428] Setting key 5a9e0bd6934ce2bb40438b9e90876ed0
I20260620 18:45:17.625380 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39437
--builtin_ntp_servers=127.15.77.84:44529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 18:45:17.707828 25301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:17.708072 25301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:17.708105 25301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:17.709254 25301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:17.709321 25301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:17.709367 25301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:45:17.710695 25301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:44529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:17.710958 25301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:17.711261 25301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:17.716204 25306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:17.716207 25307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:17.716290 25309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:17.716797 25301 server_base.cc:1034] running on GCE node
I20260620 18:45:17.717033 25301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:17.717285 25301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:17.718451 25301 hybrid_clock.cc:648] HybridClock initialized: now 1781981117718445 us; error 36 us; skew 500 ppm
I20260620 18:45:17.719806 25301 webserver.cc:460] Webserver started at http://127.15.77.67:37061/ using document root <none> and password file <none>
I20260620 18:45:17.720064 25301 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:17.720131 25301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:17.720275 25301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:17.721378 25301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "bfb116d486ed48ae811629fdb1beef10"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "76164cf0fe5bef46ee9bc04f0c482bdb"
server_key_iv: "b8e25174cb5f89912f43cc29aacf9b08"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.721750 25301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "bfb116d486ed48ae811629fdb1beef10"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "76164cf0fe5bef46ee9bc04f0c482bdb"
server_key_iv: "b8e25174cb5f89912f43cc29aacf9b08"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.723182 25301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:45:17.724009 25315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.724180 25301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:45:17.724279 25301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "bfb116d486ed48ae811629fdb1beef10"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "76164cf0fe5bef46ee9bc04f0c482bdb"
server_key_iv: "b8e25174cb5f89912f43cc29aacf9b08"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.724367 25301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:17.754726 25301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:17.755077 25301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:17.755234 25301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:17.755508 25301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:17.755858 25301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:17.755913 25301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.755954 25301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:17.755993 25301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.762365 25301 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:33509
I20260620 18:45:17.762423 25428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:33509 every 8 connection(s)
I20260620 18:45:17.762796 25301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:45:17.768309 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:17.763598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:39395 (local address 127.15.77.126:39437)
0620 18:45:17.763662 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:17.763664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:17.763926 (+ 262us) server_negotiation.cc:407] Connection header received
0620 18:45:17.764478 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:17.764480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:17.764492 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:17.764547 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:17.765329 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.765833 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.766844 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.767057 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.767133 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:17.767978 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:17.767980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:17.767980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:45:17.767985 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:17.768001 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:45:17.768005 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:17.768085 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:17.768199 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 18:45:17.768227 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:45:17.768668 25431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:17.763567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:39437 (local address 127.15.77.67:39395)
0620 18:45:17.763803 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:17.763837 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:17.764368 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:17.764664 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:17.764668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:17.764679 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:17.765226 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:17.765231 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.765936 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.765937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:17.766741 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:17.766743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:17.767161 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:17.767163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:17.767292 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:17.767833 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:45:17.767833 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:17.767863 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:45:17.767885 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:17.768088 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:17.768094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:17.768097 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:45:17.768187 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 18:45:17.768290 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260620 18:45:17.769145 25429 heartbeater.cc:347] Connected to a master server at 127.15.77.126:39437
I20260620 18:45:17.769225 25429 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:17.769372 25429 heartbeater.cc:511] Master 127.15.77.126:39437 requested a full tablet report, sending...
I20260620 18:45:17.769721 24979 ts_manager.cc:194] Registered new tserver with Master: bfb116d486ed48ae811629fdb1beef10 (127.15.77.67:33509)
I20260620 18:45:17.770051 24979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.67:39395
I20260620 18:45:17.770623 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 25301
I20260620 18:45:17.770723 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:45:17.770929 15669 external_mini_cluster.cc:1428] Setting key 5c3c66dad471c56cc4b1ea65266201f1
I20260620 18:45:17.771872 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 18:45:17.773558 25018 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:17.773109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33750 (local address 127.15.77.126:39437)
0620 18:45:17.773147 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:17.773149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:17.773272 (+ 123us) server_negotiation.cc:407] Connection header received
0620 18:45:17.773313 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:17.773313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:17.773321 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:17.773372 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:17.773474 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33750: BlockingRecv error: recv got EOF from 127.0.0.1:33750 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 18:45:17.773658 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 25039
I20260620 18:45:17.778680 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 25170
I20260620 18:45:17.783094 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 25301
I20260620 18:45:17.787429 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 24948
2026-06-20T18:45:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (622 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-20T18:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:45:17Z Disabled control of system clock
I20260620 18:45:17.814476 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34009
--webserver_interface=127.15.77.126
--webserver_port=0
--builtin_ntp_servers=127.15.77.84:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:34009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 18:45:17.896211 25440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:17.896425 25440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:17.896446 25440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:17.897533 25440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:45:17.897567 25440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:17.897579 25440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:17.897593 25440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:45:17.897604 25440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:45:17.898983 25440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:34009
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:17.899194 25440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:17.899484 25440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:17.904381 25446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:17.904420 25445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:17.904505 25440 server_base.cc:1034] running on GCE node
W20260620 18:45:17.904397 25448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:17.904923 25440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:17.905211 25440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:17.906383 25440 hybrid_clock.cc:648] HybridClock initialized: now 1781981117906359 us; error 42 us; skew 500 ppm
I20260620 18:45:17.907737 25440 webserver.cc:460] Webserver started at http://127.15.77.126:38981/ using document root <none> and password file <none>
I20260620 18:45:17.907975 25440 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:17.908022 25440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:17.908214 25440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:17.909287 25440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "960fd26c04ad404e8ed6b60153194618"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "bb859c0d92bdcf734da5ad1ca576188f"
server_key_iv: "adbc22418d5f1586fa660802d3bb0caa"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.909647 25440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "960fd26c04ad404e8ed6b60153194618"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "bb859c0d92bdcf734da5ad1ca576188f"
server_key_iv: "adbc22418d5f1586fa660802d3bb0caa"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.910976 25440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:45:17.911906 25454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:17.912086 25440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 18:45:17.912205 25440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "960fd26c04ad404e8ed6b60153194618"
format_stamp: "Formatted at 2026-06-20 18:45:17 on dist-test-slave-tvl9"
server_key: "bb859c0d92bdcf734da5ad1ca576188f"
server_key_iv: "adbc22418d5f1586fa660802d3bb0caa"
server_key_version: "encryptionkey@0"
I20260620 18:45:17.912292 25440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:17.915633 25440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:17.915905 25440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:17.916057 25440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:17.920006 25440 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:34009
I20260620 18:45:17.920034 25506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:34009 every 8 connection(s)
I20260620 18:45:17.920389 25440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:45:17.921137 25507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:45:17.924057 25507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618: Bootstrap starting.
I20260620 18:45:17.924683 25507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618: Neither blocks nor log segments found. Creating new log.
I20260620 18:45:17.924997 25507 log.cc:824] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618: Log is configured to *not* fsync() on all Append() calls
I20260620 18:45:17.925705 25507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618: No bootstrap required, opened a new log
I20260620 18:45:17.927414 25507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960fd26c04ad404e8ed6b60153194618" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34009 } }
I20260620 18:45:17.927565 25507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:45:17.927588 25507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 960fd26c04ad404e8ed6b60153194618, State: Initialized, Role: FOLLOWER
I20260620 18:45:17.927770 25507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960fd26c04ad404e8ed6b60153194618" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34009 } }
I20260620 18:45:17.927834 25507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:45:17.927877 25507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:45:17.927927 25507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:45:17.928496 25507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960fd26c04ad404e8ed6b60153194618" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34009 } }
I20260620 18:45:17.928618 25507 leader_election.cc:302] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 960fd26c04ad404e8ed6b60153194618; no voters:
I20260620 18:45:17.928848 25507 leader_election.cc:288] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:45:17.928881 25510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:45:17.928871 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 25440
I20260620 18:45:17.928984 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:45:17.928999 25510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [term 1 LEADER]: Becoming Leader. State: Replica: 960fd26c04ad404e8ed6b60153194618, State: Running, Role: LEADER
I20260620 18:45:17.929085 25510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960fd26c04ad404e8ed6b60153194618" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34009 } }
I20260620 18:45:17.929139 25507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:45:17.929152 15669 external_mini_cluster.cc:1428] Setting key 91afb627b897e559678f87368f5c32a5
I20260620 18:45:17.929414 25510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "960fd26c04ad404e8ed6b60153194618" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960fd26c04ad404e8ed6b60153194618" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34009 } } }
I20260620 18:45:17.929592 25510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [sys.catalog]: This master's current role is: LEADER
I20260620 18:45:17.929812 25510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 960fd26c04ad404e8ed6b60153194618. Latest consensus state: current_term: 1 leader_uuid: "960fd26c04ad404e8ed6b60153194618" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960fd26c04ad404e8ed6b60153194618" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34009 } } }
I20260620 18:45:17.929932 25510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618 [sys.catalog]: This master's current role is: LEADER
I20260620 18:45:17.930138 25519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:45:17.930714 25519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:45:17.932298 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:17.929797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51966 (local address 127.15.77.126:34009)
0620 18:45:17.930532 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:17.930539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:17.930566 (+ 27us) server_negotiation.cc:407] Connection header received
0620 18:45:17.930693 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:17.930700 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:17.930747 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:17.931258 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:17.931791 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:17.931794 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:17.931794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:45:17.931797 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:17.931852 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:45:17.931870 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:17.931943 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:17.932065 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 18:45:17.932132 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":75,"threads_started":1}
I20260620 18:45:17.932974 25519 catalog_manager.cc:1347] Generated new cluster ID: ddcba2efd1fe4f7589920aa17d4a6ff0
I20260620 18:45:17.933028 25519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:45:17.965605 25519 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:45:17.965821 25519 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:45:17.981349 25519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 960fd26c04ad404e8ed6b60153194618: Generated new TSK 0
I20260620 18:45:17.981747 25519 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:45:17.984911 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34009
--builtin_ntp_servers=127.15.77.84:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 18:45:18.086524 25531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:18.086750 25531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:18.086774 25531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:18.088035 25531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:18.088084 25531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:18.088111 25531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:45:18.089507 25531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:18.089766 25531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:18.090085 25531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:18.095230 25539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:18.095207 25537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:18.095224 25536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:18.095247 25531 server_base.cc:1034] running on GCE node
I20260620 18:45:18.095715 25531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:18.095938 25531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:18.097084 25531 hybrid_clock.cc:648] HybridClock initialized: now 1781981118097070 us; error 33 us; skew 500 ppm
I20260620 18:45:18.098431 25531 webserver.cc:460] Webserver started at http://127.15.77.65:44561/ using document root <none> and password file <none>
I20260620 18:45:18.098699 25531 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:18.098750 25531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:18.098925 25531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:18.100107 25531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "561fc573086642a79d287721bf38ee65"
format_stamp: "Formatted at 2026-06-20 18:45:18 on dist-test-slave-tvl9"
server_key: "365cdfae5b486053df06dc2fe4eec682"
server_key_iv: "dbe49a41e99248e1d4d78bee7610b6b1"
server_key_version: "encryptionkey@0"
I20260620 18:45:18.100508 25531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "561fc573086642a79d287721bf38ee65"
format_stamp: "Formatted at 2026-06-20 18:45:18 on dist-test-slave-tvl9"
server_key: "365cdfae5b486053df06dc2fe4eec682"
server_key_iv: "dbe49a41e99248e1d4d78bee7610b6b1"
server_key_version: "encryptionkey@0"
I20260620 18:45:18.101855 25531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 18:45:18.102605 25545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:18.102809 25531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:45:18.102916 25531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "561fc573086642a79d287721bf38ee65"
format_stamp: "Formatted at 2026-06-20 18:45:18 on dist-test-slave-tvl9"
server_key: "365cdfae5b486053df06dc2fe4eec682"
server_key_iv: "dbe49a41e99248e1d4d78bee7610b6b1"
server_key_version: "encryptionkey@0"
I20260620 18:45:18.102990 25531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:18.106848 25531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:18.107149 25531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:18.107301 25531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:18.107559 25531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:18.107872 25531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:18.107925 25531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:18.107964 25531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:18.107992 25531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:18.113593 25531 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:45741
I20260620 18:45:18.113682 25658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:45741 every 8 connection(s)
I20260620 18:45:18.114014 25531 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:45:18.117417 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:18.114792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:52561 (local address 127.15.77.126:34009)
0620 18:45:18.114862 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:18.114863 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:18.115145 (+ 282us) server_negotiation.cc:407] Connection header received
0620 18:45:18.115756 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:18.115758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:18.115767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:18.115814 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:18.117112 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:18.117114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:18.117114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:45:18.117119 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:18.117136 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:45:18.117141 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:18.117222 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:18.117327 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 18:45:18.117345 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 18:45:18.117624 25661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:18.114763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:34009 (local address 127.15.77.65:52561)
0620 18:45:18.115013 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:18.115049 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:18.115628 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:18.115920 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:18.115925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:18.115935 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:18.116946 (+ 1011us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:45:18.116947 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:18.116976 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:45:18.117006 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:18.117219 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:18.117228 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:18.117231 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:45:18.117339 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 18:45:18.117403 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":101,"threads_started":1}
I20260620 18:45:18.118122 25659 heartbeater.cc:347] Connected to a master server at 127.15.77.126:34009
I20260620 18:45:18.118227 25659 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:18.118382 25659 heartbeater.cc:511] Master 127.15.77.126:34009 requested a full tablet report, sending...
I20260620 18:45:18.118909 25471 ts_manager.cc:194] Registered new tserver with Master: 561fc573086642a79d287721bf38ee65 (127.15.77.65:45741)
I20260620 18:45:18.120921 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 25531
I20260620 18:45:18.121024 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:45:18.121225 15669 external_mini_cluster.cc:1428] Setting key 1c76f58471624a79f52cf605cec4eca8
I20260620 18:45:18.122174 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34009
--builtin_ntp_servers=127.15.77.84:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 18:45:18.204200 25662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:18.204423 25662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:18.204447 25662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:18.205562 25662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:18.205598 25662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:18.205623 25662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:45:18.207273 25662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:18.207512 25662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:18.207780 25662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:18.212694 25667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:18.212824 25662 server_base.cc:1034] running on GCE node
W20260620 18:45:18.212764 25668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:18.212913 25670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:18.213155 25662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:18.213411 25662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:18.214576 25662 hybrid_clock.cc:648] HybridClock initialized: now 1781981118214570 us; error 39 us; skew 500 ppm
I20260620 18:45:18.216080 25662 webserver.cc:460] Webserver started at http://127.15.77.66:46849/ using document root <none> and password file <none>
I20260620 18:45:18.216342 25662 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:18.216413 25662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:18.216565 25662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:18.217655 25662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "e9bea076bb2e4947b4d98389fa815855"
format_stamp: "Formatted at 2026-06-20 18:45:18 on dist-test-slave-tvl9"
server_key: "57fc7704b1bd3ff2760fa3c13b1558ec"
server_key_iv: "282f219ed345ca8b5a6a5d30bcb2f0b6"
server_key_version: "encryptionkey@0"
I20260620 18:45:18.218027 25662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "e9bea076bb2e4947b4d98389fa815855"
format_stamp: "Formatted at 2026-06-20 18:45:18 on dist-test-slave-tvl9"
server_key: "57fc7704b1bd3ff2760fa3c13b1558ec"
server_key_iv: "282f219ed345ca8b5a6a5d30bcb2f0b6"
server_key_version: "encryptionkey@0"
I20260620 18:45:18.219384 25662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260620 18:45:18.220127 25676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:18.220293 25662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:45:18.220394 25662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "e9bea076bb2e4947b4d98389fa815855"
format_stamp: "Formatted at 2026-06-20 18:45:18 on dist-test-slave-tvl9"
server_key: "57fc7704b1bd3ff2760fa3c13b1558ec"
server_key_iv: "282f219ed345ca8b5a6a5d30bcb2f0b6"
server_key_version: "encryptionkey@0"
I20260620 18:45:18.220479 25662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:18.223879 25662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:18.224148 25662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:18.224289 25662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:18.224491 25662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:18.224805 25662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:18.224860 25662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:18.224900 25662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:18.224929 25662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:18.230960 25662 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:44521
I20260620 18:45:18.231002 25789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:44521 every 8 connection(s)
I20260620 18:45:18.231467 25662 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:45:18.234652 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:18.232234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:45327 (local address 127.15.77.126:34009)
0620 18:45:18.232283 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:18.232285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:18.232559 (+ 274us) server_negotiation.cc:407] Connection header received
0620 18:45:18.233089 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:18.233091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:18.233097 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:18.233144 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:18.234381 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:18.234382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:18.234382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:45:18.234387 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:18.234403 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:45:18.234408 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:18.234463 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:18.234564 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 18:45:18.234579 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:45:18.234894 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:18.232226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:34009 (local address 127.15.77.66:45327)
0620 18:45:18.232436 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:18.232472 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:18.232980 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:18.233217 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:18.233222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:18.233238 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:18.234220 (+ 982us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:45:18.234220 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:18.234249 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:45:18.234277 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:18.234474 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:18.234479 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:18.234482 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:45:18.234574 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 18:45:18.234637 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260620 18:45:18.235389 25790 heartbeater.cc:347] Connected to a master server at 127.15.77.126:34009
I20260620 18:45:18.235482 25790 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:18.235597 25790 heartbeater.cc:511] Master 127.15.77.126:34009 requested a full tablet report, sending...
I20260620 18:45:18.235949 25471 ts_manager.cc:194] Registered new tserver with Master: e9bea076bb2e4947b4d98389fa815855 (127.15.77.66:44521)
I20260620 18:45:18.237363 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 25662
I20260620 18:45:18.237447 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:45:18.237594 15669 external_mini_cluster.cc:1428] Setting key 7dd65d2e9b9715d85c2589eb113f72c6
I20260620 18:45:18.238488 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34009
--builtin_ntp_servers=127.15.77.84:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 18:45:18.325373 25793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:18.325623 25793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:18.325685 25793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:18.326887 25793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:18.326956 25793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:18.327004 25793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:45:18.328493 25793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:18.328784 25793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:18.329089 25793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:18.334393 25799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:18.334388 25801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:18.334460 25793 server_base.cc:1034] running on GCE node
W20260620 18:45:18.334388 25798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:18.334789 25793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:18.335049 25793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:18.336192 25793 hybrid_clock.cc:648] HybridClock initialized: now 1781981118336180 us; error 35 us; skew 500 ppm
I20260620 18:45:18.337435 25793 webserver.cc:460] Webserver started at http://127.15.77.67:35865/ using document root <none> and password file <none>
I20260620 18:45:18.337742 25793 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:18.337791 25793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:18.337934 25793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:18.339018 25793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "8d454d1bbea34965941d482133a2deae"
format_stamp: "Formatted at 2026-06-20 18:45:18 on dist-test-slave-tvl9"
server_key: "a1e02bcf6af04310e19734d6cd10b3f0"
server_key_iv: "97f3b057dc06ff029127bfbfb8ad0806"
server_key_version: "encryptionkey@0"
I20260620 18:45:18.339422 25793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "8d454d1bbea34965941d482133a2deae"
format_stamp: "Formatted at 2026-06-20 18:45:18 on dist-test-slave-tvl9"
server_key: "a1e02bcf6af04310e19734d6cd10b3f0"
server_key_iv: "97f3b057dc06ff029127bfbfb8ad0806"
server_key_version: "encryptionkey@0"
I20260620 18:45:18.340708 25793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:45:18.341411 25807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:18.341578 25793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:45:18.341681 25793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "8d454d1bbea34965941d482133a2deae"
format_stamp: "Formatted at 2026-06-20 18:45:18 on dist-test-slave-tvl9"
server_key: "a1e02bcf6af04310e19734d6cd10b3f0"
server_key_iv: "97f3b057dc06ff029127bfbfb8ad0806"
server_key_version: "encryptionkey@0"
I20260620 18:45:18.341768 25793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:18.345148 25793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:18.345412 25793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:18.345561 25793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:18.345763 25793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:18.346066 25793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:18.346119 25793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:18.346168 25793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:18.346196 25793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:18.351876 25793 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:41405
I20260620 18:45:18.351925 25920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:41405 every 8 connection(s)
I20260620 18:45:18.352257 25793 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:45:18.353292 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 25793
I20260620 18:45:18.353391 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:45:18.353586 15669 external_mini_cluster.cc:1428] Setting key 8bca01e540da693acbbd1efce73a99da
I20260620 18:45:18.355646 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:18.352920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:53255 (local address 127.15.77.126:34009)
0620 18:45:18.352966 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:18.352968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:18.353286 (+ 318us) server_negotiation.cc:407] Connection header received
0620 18:45:18.353990 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:18.353991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:18.353998 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:18.354055 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:18.355421 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:18.355422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:18.355422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:45:18.355426 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:18.355442 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:45:18.355446 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:18.355495 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:18.355575 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 18:45:18.355588 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:45:18.355830 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:18.352897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:34009 (local address 127.15.77.67:53255)
0620 18:45:18.353144 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:18.353192 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:18.353860 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:18.354209 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:18.354213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:18.354223 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:18.355190 (+ 967us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:45:18.355191 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:18.355218 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:45:18.355252 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:18.355508 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:18.355512 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:18.355515 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:45:18.355582 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 18:45:18.355631 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260620 18:45:18.356317 25921 heartbeater.cc:347] Connected to a master server at 127.15.77.126:34009
I20260620 18:45:18.356390 25921 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:18.356551 25921 heartbeater.cc:511] Master 127.15.77.126:34009 requested a full tablet report, sending...
I20260620 18:45:18.356922 25471 ts_manager.cc:194] Registered new tserver with Master: 8d454d1bbea34965941d482133a2deae (127.15.77.67:41405)
I20260620 18:45:18.365537 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 18:45:18.367558 25521 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:45:18.366968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51970 (local address 127.15.77.126:34009)
0620 18:45:18.367038 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:18.367040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:18.367173 (+ 133us) server_negotiation.cc:407] Connection header received
0620 18:45:18.367249 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:18.367249 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:18.367255 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:18.367343 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:18.367479 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51970: BlockingRecv error: recv got EOF from 127.0.0.1:51970 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:45:18.367624 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 25531
I20260620 18:45:18.372736 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 25662
I20260620 18:45:18.377698 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 25793
I20260620 18:45:18.381948 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 25440
2026-06-20T18:45:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (594 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:45:18 dist-test-slave-tvl9 krb5kdc[25931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:45:18 dist-test-slave-tvl9 krb5kdc[25931](info): set up 2 sockets
Jun 20 18:45:18 dist-test-slave-tvl9 krb5kdc[25931](info): commencing operation
krb5kdc: starting...
W20260620 18:45:19.431555 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:45:19 dist-test-slave-tvl9 krb5kdc[25931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:45:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:45:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:45:19.547976 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:45169
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:42769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:45169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:45:19.628985 25944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:19.629238 25944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:19.629276 25944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:19.630419 25944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:45:19.630488 25944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:19.630512 25944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:19.630535 25944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:45:19.630563 25944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:45:19.632045 25944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:42769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:45169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:45169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:19.632330 25944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:19.632560 25944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:19.637537 25950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:19.637617 25952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:19.637790 25944 server_base.cc:1034] running on GCE node
W20260620 18:45:19.637549 25949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:19.638118 25944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:19.638415 25944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:19.639576 25944 hybrid_clock.cc:648] HybridClock initialized: now 1781981119639561 us; error 25 us; skew 500 ppm
Jun 20 18:45:19 dist-test-slave-tvl9 krb5kdc[25931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981119, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:19.642064 25944 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:45:19.642616 25944 webserver.cc:460] Webserver started at http://127.15.77.126:43975/ using document root <none> and password file <none>
I20260620 18:45:19.642850 25944 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:19.642918 25944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:19.643074 25944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:19.644212 25944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "ee206ab6a6fd48539a302d6599d6afd4"
format_stamp: "Formatted at 2026-06-20 18:45:19 on dist-test-slave-tvl9"
server_key: "a3bbdffd8fe059e7ca39379deb2079b2"
server_key_iv: "9998b283ba9213b3fc1ad799eb6c8a8d"
server_key_version: "encryptionkey@0"
I20260620 18:45:19.644582 25944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "ee206ab6a6fd48539a302d6599d6afd4"
format_stamp: "Formatted at 2026-06-20 18:45:19 on dist-test-slave-tvl9"
server_key: "a3bbdffd8fe059e7ca39379deb2079b2"
server_key_iv: "9998b283ba9213b3fc1ad799eb6c8a8d"
server_key_version: "encryptionkey@0"
I20260620 18:45:19.645892 25944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:45:19.646634 25959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:19.646818 25944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:45:19.646921 25944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "ee206ab6a6fd48539a302d6599d6afd4"
format_stamp: "Formatted at 2026-06-20 18:45:19 on dist-test-slave-tvl9"
server_key: "a3bbdffd8fe059e7ca39379deb2079b2"
server_key_iv: "9998b283ba9213b3fc1ad799eb6c8a8d"
server_key_version: "encryptionkey@0"
I20260620 18:45:19.647009 25944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:19.665117 25944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:19.667083 25944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:19.667268 25944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:19.671378 25944 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:45169
I20260620 18:45:19.671416 26011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:45169 every 8 connection(s)
I20260620 18:45:19.671805 25944 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:45:19.672864 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 25944
I20260620 18:45:19.672969 26012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:45:19.672977 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:45:19.673197 15669 external_mini_cluster.cc:1428] Setting key 8991f5d7a5ca73cde0131db7c10a5398
I20260620 18:45:19.676235 26012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4: Bootstrap starting.
I20260620 18:45:19.676851 26012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4: Neither blocks nor log segments found. Creating new log.
I20260620 18:45:19.677146 26012 log.cc:824] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4: Log is configured to *not* fsync() on all Append() calls
I20260620 18:45:19.677817 26012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4: No bootstrap required, opened a new log
Jun 20 18:45:19 dist-test-slave-tvl9 krb5kdc[25931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:45:19.680173 26012 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee206ab6a6fd48539a302d6599d6afd4" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45169 } }
I20260620 18:45:19.680361 26012 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:45:19.680401 26012 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee206ab6a6fd48539a302d6599d6afd4, State: Initialized, Role: FOLLOWER
I20260620 18:45:19.680552 26012 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee206ab6a6fd48539a302d6599d6afd4" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45169 } }
I20260620 18:45:19.680661 26012 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:45:19.680697 26012 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:45:19.680753 26012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:45:19.681526 26012 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee206ab6a6fd48539a302d6599d6afd4" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45169 } }
I20260620 18:45:19.681658 26012 leader_election.cc:302] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee206ab6a6fd48539a302d6599d6afd4; no voters:
I20260620 18:45:19.681826 26012 leader_election.cc:288] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:45:19.681890 26017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:45:19.682092 26012 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:45:19.682133 26017 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [term 1 LEADER]: Becoming Leader. State: Replica: ee206ab6a6fd48539a302d6599d6afd4, State: Running, Role: LEADER
I20260620 18:45:19.682336 26017 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee206ab6a6fd48539a302d6599d6afd4" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45169 } }
I20260620 18:45:19.682690 26018 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee206ab6a6fd48539a302d6599d6afd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee206ab6a6fd48539a302d6599d6afd4" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45169 } } }
I20260620 18:45:19.682777 26018 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [sys.catalog]: This master's current role is: LEADER
I20260620 18:45:19.682958 26019 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee206ab6a6fd48539a302d6599d6afd4. Latest consensus state: current_term: 1 leader_uuid: "ee206ab6a6fd48539a302d6599d6afd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee206ab6a6fd48539a302d6599d6afd4" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 45169 } } }
I20260620 18:45:19.683023 26019 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4 [sys.catalog]: This master's current role is: LEADER
I20260620 18:45:19.683008 26026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:45:19.683722 26026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:45:19.684718 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:19.673733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.15.77.126:45169)
0620 18:45:19.673924 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:19.673933 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:19.673957 (+ 24us) server_negotiation.cc:407] Connection header received
0620 18:45:19.674472 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:19.674479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:19.674526 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:19.674640 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:19.675423 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:19.676342 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:19.677178 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:19.677640 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:19.677765 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:19.680506 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:19.680510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:19.680511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:45:19.680524 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:19.682609 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:19.683069 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:19.683072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:19.683072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:19.683179 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:19.683434 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:19.683435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:19.683436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:19.683768 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:19.684310 (+ 542us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:19.684382 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 18:45:19.684516 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260620 18:45:19.685518 26026 catalog_manager.cc:1347] Generated new cluster ID: aa609f2334a84e10a8e9556e0766800e
I20260620 18:45:19.685557 26026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:45:19.693243 26026 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:45:19.693775 26026 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:45:19.704268 26026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4: Generated new TSK 0
I20260620 18:45:19.704479 26026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:45:19.735271 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45169
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:42769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:45:19.819613 26038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:19.819872 26038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:19.819913 26038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:19.821130 26038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:19.821208 26038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:19.821254 26038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:45:19.822788 26038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:42769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:19.823081 26038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:19.823361 26038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:45:19.828589 26038 server_base.cc:1034] running on GCE node
W20260620 18:45:19.828613 26044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:19.828698 26043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:19.828685 26046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:19.829030 26038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:19.829265 26038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:19.830415 26038 hybrid_clock.cc:648] HybridClock initialized: now 1781981119830405 us; error 27 us; skew 500 ppm
Jun 20 18:45:19 dist-test-slave-tvl9 krb5kdc[25931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981119, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:19.833024 26038 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:45:19.833537 26038 webserver.cc:460] Webserver started at http://127.15.77.65:38121/ using document root <none> and password file <none>
I20260620 18:45:19.833781 26038 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:19.833854 26038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:19.834015 26038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:19.835107 26038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "e02ced161817401a98a95608351a687a"
format_stamp: "Formatted at 2026-06-20 18:45:19 on dist-test-slave-tvl9"
server_key: "167c22182d093d8c61b052e6ad6e20ea"
server_key_iv: "6f076bed868ada41006393b81837516c"
server_key_version: "encryptionkey@0"
I20260620 18:45:19.835507 26038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "e02ced161817401a98a95608351a687a"
format_stamp: "Formatted at 2026-06-20 18:45:19 on dist-test-slave-tvl9"
server_key: "167c22182d093d8c61b052e6ad6e20ea"
server_key_iv: "6f076bed868ada41006393b81837516c"
server_key_version: "encryptionkey@0"
I20260620 18:45:19.836879 26038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:45:19.837697 26053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:19.837898 26038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 18:45:19.838008 26038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "e02ced161817401a98a95608351a687a"
format_stamp: "Formatted at 2026-06-20 18:45:19 on dist-test-slave-tvl9"
server_key: "167c22182d093d8c61b052e6ad6e20ea"
server_key_iv: "6f076bed868ada41006393b81837516c"
server_key_version: "encryptionkey@0"
I20260620 18:45:19.838084 26038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:19.860205 26038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:19.862226 26038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:19.862396 26038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:19.862686 26038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:19.863020 26038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:19.863075 26038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:19.863121 26038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:19.863147 26038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:19.869110 26038 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:38051
I20260620 18:45:19.869151 26166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:38051 every 8 connection(s)
I20260620 18:45:19.869499 26038 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:45:19.870831 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 26038
I20260620 18:45:19.870941 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:45:19.871152 15669 external_mini_cluster.cc:1428] Setting key 3c560832072317a64b9a78cc87440ac0
Jun 20 18:45:19 dist-test-slave-tvl9 krb5kdc[25931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981119, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:45:19.879949 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:19.870286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:41635 (local address 127.15.77.126:45169)
0620 18:45:19.870356 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:19.870358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:19.870559 (+ 201us) server_negotiation.cc:407] Connection header received
0620 18:45:19.871179 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:19.871180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:19.871194 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:19.871232 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:19.872378 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:19.872889 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:19.874062 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:19.874396 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:19.874468 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:19.876747 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:19.876748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:19.876749 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:45:19.876758 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:19.878459 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:19.878866 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:19.878868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:19.878868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:19.878937 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:19.879185 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:19.879187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:19.879187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:19.879383 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:19.879479 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:19.879729 (+ 250us) server_negotiation.cc:299] Negotiation successful
0620 18:45:19.879818 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 18:45:19.880059 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:19.870282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:45169 (local address 127.15.77.65:41635)
0620 18:45:19.870438 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:19.870481 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:19.871049 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:19.871303 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:19.871307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:19.872061 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:19.872265 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:19.872270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:19.872996 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:19.872997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:19.873964 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:19.873966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:19.874490 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:19.874491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:19.874607 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:19.875168 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:19.875178 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:19.876648 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:19.878550 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:19.878557 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:19.878558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:19.878785 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:19.879011 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:19.879012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:19.879012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:19.879107 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:19.879465 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:19.879470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:19.879590 (+ 120us) client_negotiation.cc:769] Sending connection context
0620 18:45:19.879679 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 18:45:19.879804 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
I20260620 18:45:19.880743 26167 heartbeater.cc:347] Connected to a master server at 127.15.77.126:45169
I20260620 18:45:19.880852 26167 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:19.881081 26167 heartbeater.cc:511] Master 127.15.77.126:45169 requested a full tablet report, sending...
I20260620 18:45:19.881589 25976 ts_manager.cc:194] Registered new tserver with Master: e02ced161817401a98a95608351a687a (127.15.77.65:38051)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
I20260620 18:45:19.882491 25976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:41635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:45:19.898389 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45169
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:42769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:45:19.981061 26172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:19.981333 26172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:19.981375 26172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:19.982681 26172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:19.982746 26172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:19.982795 26172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:45:19.984418 26172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:42769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:19.984714 26172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:19.984954 26172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:19.989995 26177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:19.990159 26180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:19.990052 26172 server_base.cc:1034] running on GCE node
W20260620 18:45:19.990204 26178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:19.990554 26172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:19.990792 26172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:19.991967 26172 hybrid_clock.cc:648] HybridClock initialized: now 1781981119991946 us; error 33 us; skew 500 ppm
Jun 20 18:45:19 dist-test-slave-tvl9 krb5kdc[25931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981119, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:19.994530 26172 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:45:19.994979 26172 webserver.cc:460] Webserver started at http://127.15.77.66:38931/ using document root <none> and password file <none>
I20260620 18:45:19.995229 26172 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:19.995276 26172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:19.995466 26172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:19.996891 26172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "37e1c088eb374185a0f2dec6b48cf109"
format_stamp: "Formatted at 2026-06-20 18:45:19 on dist-test-slave-tvl9"
server_key: "6db266afe6e583bd3a98b67c1f9b3588"
server_key_iv: "7139b43da929c2a7ba10297362342a22"
server_key_version: "encryptionkey@0"
I20260620 18:45:19.997337 26172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "37e1c088eb374185a0f2dec6b48cf109"
format_stamp: "Formatted at 2026-06-20 18:45:19 on dist-test-slave-tvl9"
server_key: "6db266afe6e583bd3a98b67c1f9b3588"
server_key_iv: "7139b43da929c2a7ba10297362342a22"
server_key_version: "encryptionkey@0"
I20260620 18:45:19.999156 26172 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260620 18:45:20.000002 26187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:20.000213 26172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:45:20.000308 26172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "37e1c088eb374185a0f2dec6b48cf109"
format_stamp: "Formatted at 2026-06-20 18:45:19 on dist-test-slave-tvl9"
server_key: "6db266afe6e583bd3a98b67c1f9b3588"
server_key_iv: "7139b43da929c2a7ba10297362342a22"
server_key_version: "encryptionkey@0"
I20260620 18:45:20.000380 26172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:20.014922 26172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:20.016861 26172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:20.017027 26172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:20.017257 26172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:20.017573 26172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:20.017629 26172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:20.017670 26172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:20.017791 26172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:20.023720 26172 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:39999
I20260620 18:45:20.023793 26300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:39999 every 8 connection(s)
I20260620 18:45:20.024135 26172 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
Jun 20 18:45:20 dist-test-slave-tvl9 krb5kdc[25931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981119, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:45:20.033069 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.024964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:40927 (local address 127.15.77.126:45169)
0620 18:45:20.025113 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:20.025115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:20.025171 (+ 56us) server_negotiation.cc:407] Connection header received
0620 18:45:20.025661 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:20.025662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:20.025672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:20.025728 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:20.026640 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.027124 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.028181 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.028372 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.028426 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.030515 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:20.030517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:20.030518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:45:20.030526 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:20.031874 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:20.032253 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:20.032253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:20.032254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:20.032304 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:20.032536 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:20.032537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:20.032537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:20.032642 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:20.032696 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:20.032930 (+ 234us) server_negotiation.cc:299] Negotiation successful
0620 18:45:20.033002 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260620 18:45:20.033246 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.024860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:45169 (local address 127.15.77.66:40927)
0620 18:45:20.025038 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:20.025083 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:20.025572 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:20.025789 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:20.025792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:20.026362 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:20.026560 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.026565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.027205 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.027206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.028095 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.028098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.028446 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.028447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.028558 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.029040 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:20.029049 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:20.030417 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:20.031951 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:20.031957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:20.031959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:20.032186 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:20.032368 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:20.032369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:20.032370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:20.032481 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:20.032699 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:20.032704 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:20.032821 (+ 117us) client_negotiation.cc:769] Sending connection context
0620 18:45:20.032905 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 18:45:20.033026 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260620 18:45:20.033596 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 26172
I20260620 18:45:20.033703 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:45:20.033771 26301 heartbeater.cc:347] Connected to a master server at 127.15.77.126:45169
I20260620 18:45:20.033874 26301 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:20.033937 15669 external_mini_cluster.cc:1428] Setting key 47984c85cccfa99710b29c5635b11fa2
I20260620 18:45:20.034093 26301 heartbeater.cc:511] Master 127.15.77.126:45169 requested a full tablet report, sending...
I20260620 18:45:20.034509 25975 ts_manager.cc:194] Registered new tserver with Master: 37e1c088eb374185a0f2dec6b48cf109 (127.15.77.66:39999)
I20260620 18:45:20.034917 25975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:40927
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:45:20.058648 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45169
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:42769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:45:20.141160 26306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:45:20.141446 26306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:45:20.141486 26306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:45:20.142721 26306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:45:20.142793 26306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:45:20.142839 26306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:45:20.144507 26306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:42769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:45169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:45:20.144948 26306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:45:20.145234 26306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:45:20.150550 26312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:20.150552 26314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:45:20.150545 26311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:45:20.150975 26306 server_base.cc:1034] running on GCE node
I20260620 18:45:20.151156 26306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:45:20.151437 26306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:45:20.152585 26306 hybrid_clock.cc:648] HybridClock initialized: now 1781981120152568 us; error 31 us; skew 500 ppm
Jun 20 18:45:20 dist-test-slave-tvl9 krb5kdc[25931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981120, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:45:20.155092 26306 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:45:20.155623 26306 webserver.cc:460] Webserver started at http://127.15.77.67:40613/ using document root <none> and password file <none>
I20260620 18:45:20.155884 26306 fs_manager.cc:359] Metadata directory not provided
I20260620 18:45:20.155952 26306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:45:20.156108 26306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:45:20.157183 26306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "c02e9c5c52a942148af6ca4f9935f499"
format_stamp: "Formatted at 2026-06-20 18:45:20 on dist-test-slave-tvl9"
server_key: "76287c5ecbe68bec5053b5c128b278b7"
server_key_iv: "eae2914e23ef06c399990a2ffc23e993"
server_key_version: "encryptionkey@0"
I20260620 18:45:20.157548 26306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "c02e9c5c52a942148af6ca4f9935f499"
format_stamp: "Formatted at 2026-06-20 18:45:20 on dist-test-slave-tvl9"
server_key: "76287c5ecbe68bec5053b5c128b278b7"
server_key_iv: "eae2914e23ef06c399990a2ffc23e993"
server_key_version: "encryptionkey@0"
I20260620 18:45:20.158882 26306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:45:20.159691 26321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:20.159866 26306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:20.159976 26306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "c02e9c5c52a942148af6ca4f9935f499"
format_stamp: "Formatted at 2026-06-20 18:45:20 on dist-test-slave-tvl9"
server_key: "76287c5ecbe68bec5053b5c128b278b7"
server_key_iv: "eae2914e23ef06c399990a2ffc23e993"
server_key_version: "encryptionkey@0"
I20260620 18:45:20.160053 26306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:45:20.185868 26306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:45:20.187949 26306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:45:20.188068 26306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:45:20.188267 26306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:45:20.188591 26306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:45:20.188625 26306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:20.188684 26306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:45:20.188715 26306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:45:20.194847 26306 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:32799
I20260620 18:45:20.194887 26434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:32799 every 8 connection(s)
I20260620 18:45:20.195245 26306 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
Jun 20 18:45:20 dist-test-slave-tvl9 krb5kdc[25931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981120, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:45:20.203871 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 26306
I20260620 18:45:20.203987 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:45:20.204201 15669 external_mini_cluster.cc:1428] Setting key 5c025674e1cca1c67a799feb0298529d
I20260620 18:45:20.205235 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.196024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:53207 (local address 127.15.77.126:45169)
0620 18:45:20.196069 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:20.196071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:20.196335 (+ 264us) server_negotiation.cc:407] Connection header received
0620 18:45:20.196909 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:20.196911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:20.196924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:20.196976 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:20.197953 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.198471 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.199644 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.199840 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.199941 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.202064 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:20.202065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:20.202066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:45:20.202073 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:20.203409 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:20.203885 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:20.203886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:20.203886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:20.203960 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:20.204436 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:20.204437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:20.204438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:20.204586 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:20.204642 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:20.205024 (+ 382us) server_negotiation.cc:299] Negotiation successful
0620 18:45:20.205128 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:45:20.205498 26437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.196020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:45169 (local address 127.15.77.67:53207)
0620 18:45:20.196203 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:20.196241 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:20.196782 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:20.197047 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:20.197052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:20.197647 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:45:20.197843 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.197849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.198573 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.198574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.199541 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.199544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.199949 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.199950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.200090 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.200577 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:45:20.200599 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:45:20.201977 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:45:20.203478 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:20.203484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:20.203486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:20.203803 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:20.204211 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:45:20.204212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:45:20.204212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:45:20.204366 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:45:20.204703 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:45:20.204708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:45:20.204888 (+ 180us) client_negotiation.cc:769] Sending connection context
0620 18:45:20.205012 (+ 124us) client_negotiation.cc:240] Negotiation successful
0620 18:45:20.205227 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260620 18:45:20.205981 26435 heartbeater.cc:347] Connected to a master server at 127.15.77.126:45169
I20260620 18:45:20.206090 26435 heartbeater.cc:464] Registering TS with master...
I20260620 18:45:20.206286 26435 heartbeater.cc:511] Master 127.15.77.126:45169 requested a full tablet report, sending...
I20260620 18:45:20.206682 25975 ts_manager.cc:194] Registered new tserver with Master: c02e9c5c52a942148af6ca4f9935f499 (127.15.77.67:32799)
I20260620 18:45:20.207018 25975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:53207
I20260620 18:45:20.216351 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:45:20.223814 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.217678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46072 (local address 127.15.77.126:45169)
0620 18:45:20.217757 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:20.217760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:20.217837 (+ 77us) server_negotiation.cc:407] Connection header received
0620 18:45:20.217929 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:20.217930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:20.217941 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:20.218003 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:45:20.218577 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.219079 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.219909 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.220109 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.220203 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.221452 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:45:20.221453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:45:20.221453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:45:20.221461 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:45:20.222639 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:20.222956 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:20.222956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:20.222957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:20.223008 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:45:20.223173 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:45:20.223174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:45:20.223174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:45:20.223464 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:45:20.223557 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:20.223683 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 18:45:20.223745 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 18:45:20.225656 25975 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:45:20.227291 25975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:45:20.234094 26448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.229792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52828 (local address 127.15.77.67:32799)
0620 18:45:20.229940 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:20.229945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:20.230175 (+ 230us) server_negotiation.cc:407] Connection header received
0620 18:45:20.230223 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:20.230225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:20.230242 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:20.230312 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:45:20.230529 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.231699 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.232869 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.233628 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.233704 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.233763 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:20.233942 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 18:45:20.233987 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260620 18:45:20.234068 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.229706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:32799 (local address 127.0.0.1:52828)
0620 18:45:20.230002 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:20.230031 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:20.230111 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:20.230381 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:20.230382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:20.230383 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:45:20.230448 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.230449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.231813 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.231814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.232769 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.232772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.233727 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.233728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.233824 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.233839 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 18:45:20.233913 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 18:45:20.233973 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":102,"thread_start_us":92,"threads_started":1}
I20260620 18:45:20.234215 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.229898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:38051 (local address 127.0.0.1:47368)
0620 18:45:20.230419 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:20.230452 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:20.230502 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:20.230772 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:20.230773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:20.230775 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:45:20.230835 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.230836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.231758 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.231759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.233138 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.233140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.233907 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.233908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.233971 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.233975 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:45:20.234020 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 18:45:20.234053 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":109,"thread_start_us":69,"threads_started":1}
I20260620 18:45:20.234230 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.230124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47368 (local address 127.15.77.65:38051)
0620 18:45:20.230347 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:20.230351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:20.230560 (+ 209us) server_negotiation.cc:407] Connection header received
0620 18:45:20.230604 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:20.230607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:20.230622 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:20.230690 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:45:20.230909 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.231663 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.233208 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.233748 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.233818 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.233874 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:20.234108 (+ 234us) server_negotiation.cc:299] Negotiation successful
0620 18:45:20.234155 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":110,"threads_started":1}
I20260620 18:45:20.234505 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.229704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:39999 (local address 127.0.0.1:36268)
0620 18:45:20.229978 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:20.230020 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:20.230085 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:20.230341 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:20.230344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:20.230346 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:45:20.230442 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.230444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.231829 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.231830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.233150 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.233152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.234243 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.234244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.234339 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.234344 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:45:20.234382 (+ 38us) client_negotiation.cc:240] Negotiation successful
0620 18:45:20.234428 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":103,"thread_start_us":58,"threads_started":1}
I20260620 18:45:20.234835 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.229792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36268 (local address 127.15.77.66:39999)
0620 18:45:20.230150 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:20.230155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:20.230166 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:45:20.230203 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:20.230204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:20.230218 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:20.230328 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:45:20.230532 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.231699 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.233240 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.233984 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.234616 (+ 632us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.234692 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:20.234718 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 18:45:20.234759 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":88,"threads_started":1}
I20260620 18:45:20.235545 26101 tablet_service.cc:1467] Processing CreateTablet for tablet c01784f345a24281b618fe067c036cc4 (DEFAULT_TABLE table=test-table [id=97d3ea65be294f0c88f89b2c9a3f20db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:45:20.235637 26369 tablet_service.cc:1467] Processing CreateTablet for tablet c01784f345a24281b618fe067c036cc4 (DEFAULT_TABLE table=test-table [id=97d3ea65be294f0c88f89b2c9a3f20db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:45:20.235706 26235 tablet_service.cc:1467] Processing CreateTablet for tablet c01784f345a24281b618fe067c036cc4 (DEFAULT_TABLE table=test-table [id=97d3ea65be294f0c88f89b2c9a3f20db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:45:20.235960 26369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c01784f345a24281b618fe067c036cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:45:20.235954 26101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c01784f345a24281b618fe067c036cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:45:20.235991 26235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c01784f345a24281b618fe067c036cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:45:20.238446 26452 tablet_bootstrap.cc:492] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: Bootstrap starting.
I20260620 18:45:20.239063 26452 tablet_bootstrap.cc:654] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: Neither blocks nor log segments found. Creating new log.
I20260620 18:45:20.239467 26453 tablet_bootstrap.cc:492] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: Bootstrap starting.
I20260620 18:45:20.239708 26452 log.cc:824] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: Log is configured to *not* fsync() on all Append() calls
I20260620 18:45:20.239724 26454 tablet_bootstrap.cc:492] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a: Bootstrap starting.
I20260620 18:45:20.240157 26453 tablet_bootstrap.cc:654] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: Neither blocks nor log segments found. Creating new log.
I20260620 18:45:20.240337 26454 tablet_bootstrap.cc:654] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a: Neither blocks nor log segments found. Creating new log.
I20260620 18:45:20.240527 26452 tablet_bootstrap.cc:492] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: No bootstrap required, opened a new log
I20260620 18:45:20.240622 26452 ts_tablet_manager.cc:1397] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:45:20.240732 26453 log.cc:824] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: Log is configured to *not* fsync() on all Append() calls
I20260620 18:45:20.241003 26454 log.cc:824] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a: Log is configured to *not* fsync() on all Append() calls
I20260620 18:45:20.241518 26453 tablet_bootstrap.cc:492] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: No bootstrap required, opened a new log
I20260620 18:45:20.241626 26453 ts_tablet_manager.cc:1397] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:45:20.241823 26454 tablet_bootstrap.cc:492] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a: No bootstrap required, opened a new log
I20260620 18:45:20.241920 26454 ts_tablet_manager.cc:1397] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:45:20.242499 26452 raft_consensus.cc:348] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e1c088eb374185a0f2dec6b48cf109" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39999 } } peers { permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 } } peers { permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 } }
I20260620 18:45:20.242657 26452 raft_consensus.cc:374] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:45:20.242727 26452 raft_consensus.cc:729] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c02e9c5c52a942148af6ca4f9935f499, State: Initialized, Role: FOLLOWER
I20260620 18:45:20.242854 26452 consensus_queue.cc:260] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e1c088eb374185a0f2dec6b48cf109" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39999 } } peers { permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 } } peers { permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 } }
I20260620 18:45:20.243135 26452 ts_tablet_manager.cc:1428] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 18:45:20.243278 26435 heartbeater.cc:503] Master 127.15.77.126:45169 was elected leader, sending a full tablet report...
I20260620 18:45:20.243639 26453 raft_consensus.cc:348] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e1c088eb374185a0f2dec6b48cf109" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39999 } } peers { permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 } } peers { permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 } }
I20260620 18:45:20.243803 26453 raft_consensus.cc:374] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:45:20.243836 26453 raft_consensus.cc:729] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e1c088eb374185a0f2dec6b48cf109, State: Initialized, Role: FOLLOWER
I20260620 18:45:20.243986 26453 consensus_queue.cc:260] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e1c088eb374185a0f2dec6b48cf109" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39999 } } peers { permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 } } peers { permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 } }
I20260620 18:45:20.244307 26453 ts_tablet_manager.cc:1428] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 18:45:20.244324 26301 heartbeater.cc:503] Master 127.15.77.126:45169 was elected leader, sending a full tablet report...
I20260620 18:45:20.244381 26454 raft_consensus.cc:348] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e1c088eb374185a0f2dec6b48cf109" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39999 } } peers { permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 } } peers { permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 } }
I20260620 18:45:20.244527 26454 raft_consensus.cc:374] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:45:20.244637 26454 raft_consensus.cc:729] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02ced161817401a98a95608351a687a, State: Initialized, Role: FOLLOWER
I20260620 18:45:20.244750 26454 consensus_queue.cc:260] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e1c088eb374185a0f2dec6b48cf109" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39999 } } peers { permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 } } peers { permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 } }
I20260620 18:45:20.245054 26454 ts_tablet_manager.cc:1428] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:45:20.245067 26167 heartbeater.cc:503] Master 127.15.77.126:45169 was elected leader, sending a full tablet report...
W20260620 18:45:20.274689 26302 tablet.cc:2369] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:45:20.360622 26459 raft_consensus.cc:482] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:45:20.360788 26459 raft_consensus.cc:504] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e1c088eb374185a0f2dec6b48cf109" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39999 } } peers { permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 } } peers { permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 } }
I20260620 18:45:20.361228 26459 leader_election.cc:288] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c02e9c5c52a942148af6ca4f9935f499 (127.15.77.67:32799), e02ced161817401a98a95608351a687a (127.15.77.65:38051)
I20260620 18:45:20.365476 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.361579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:43243 (local address 127.15.77.65:38051)
0620 18:45:20.361624 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:20.361626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:20.361794 (+ 168us) server_negotiation.cc:407] Connection header received
0620 18:45:20.361830 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:20.361831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:20.361839 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:20.361907 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:45:20.362054 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.362734 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.364462 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.365051 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.365136 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.365146 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:20.365361 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 18:45:20.365402 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:45:20.365492 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.361411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:32799 (local address 127.15.77.66:55445)
0620 18:45:20.361471 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:20.361484 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:20.361527 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:20.361752 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:20.361753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:20.361755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:45:20.361830 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.361833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.362844 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.362845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.364331 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.364335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.365174 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.365175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.365268 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.365270 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:45:20.365334 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 18:45:20.365396 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":135}
I20260620 18:45:20.365476 26448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.361549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:55445 (local address 127.15.77.67:32799)
0620 18:45:20.361609 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:20.361611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:20.361622 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:45:20.361651 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:20.361652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:20.361669 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:20.361723 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:45:20.361903 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.362719 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.364463 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.365064 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.365137 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.365146 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:20.365361 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 18:45:20.365403 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 18:45:20.365499 26461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.361484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:38051 (local address 127.15.77.66:43243)
0620 18:45:20.361660 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:20.361670 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:45:20.361754 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:45:20.361922 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:45:20.361923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:45:20.361924 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:45:20.361985 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.361986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.362844 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.362845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.364331 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:45:20.364335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.365178 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.365179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:45:20.365265 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.365270 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:45:20.365334 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 18:45:20.365394 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":69,"thread_start_us":110,"threads_started":1}
I20260620 18:45:20.365945 26389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01784f345a24281b618fe067c036cc4" candidate_uuid: "37e1c088eb374185a0f2dec6b48cf109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c02e9c5c52a942148af6ca4f9935f499" is_pre_election: true
I20260620 18:45:20.365952 26121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01784f345a24281b618fe067c036cc4" candidate_uuid: "37e1c088eb374185a0f2dec6b48cf109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02ced161817401a98a95608351a687a" is_pre_election: true
I20260620 18:45:20.366129 26121 raft_consensus.cc:2454] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37e1c088eb374185a0f2dec6b48cf109 in term 0.
I20260620 18:45:20.366129 26389 raft_consensus.cc:2454] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37e1c088eb374185a0f2dec6b48cf109 in term 0.
I20260620 18:45:20.366375 26188 leader_election.cc:302] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e1c088eb374185a0f2dec6b48cf109, e02ced161817401a98a95608351a687a; no voters:
I20260620 18:45:20.366524 26459 raft_consensus.cc:2781] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:45:20.366581 26459 raft_consensus.cc:482] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:45:20.366606 26459 raft_consensus.cc:3037] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:45:20.367286 26459 raft_consensus.cc:504] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e1c088eb374185a0f2dec6b48cf109" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39999 } } peers { permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 } } peers { permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 } }
I20260620 18:45:20.367466 26459 leader_election.cc:288] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [CANDIDATE]: Term 1 election: Requested vote from peers c02e9c5c52a942148af6ca4f9935f499 (127.15.77.67:32799), e02ced161817401a98a95608351a687a (127.15.77.65:38051)
I20260620 18:45:20.367640 26389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01784f345a24281b618fe067c036cc4" candidate_uuid: "37e1c088eb374185a0f2dec6b48cf109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c02e9c5c52a942148af6ca4f9935f499"
I20260620 18:45:20.367681 26121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01784f345a24281b618fe067c036cc4" candidate_uuid: "37e1c088eb374185a0f2dec6b48cf109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02ced161817401a98a95608351a687a"
I20260620 18:45:20.367745 26389 raft_consensus.cc:3037] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:45:20.367753 26121 raft_consensus.cc:3037] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:45:20.368484 26389 raft_consensus.cc:2454] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37e1c088eb374185a0f2dec6b48cf109 in term 1.
I20260620 18:45:20.368537 26121 raft_consensus.cc:2454] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37e1c088eb374185a0f2dec6b48cf109 in term 1.
I20260620 18:45:20.368703 26188 leader_election.cc:302] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e1c088eb374185a0f2dec6b48cf109, e02ced161817401a98a95608351a687a; no voters:
I20260620 18:45:20.368832 26459 raft_consensus.cc:2781] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:45:20.369056 26459 raft_consensus.cc:686] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 1 LEADER]: Becoming Leader. State: Replica: 37e1c088eb374185a0f2dec6b48cf109, State: Running, Role: LEADER
I20260620 18:45:20.369184 26459 consensus_queue.cc:237] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e1c088eb374185a0f2dec6b48cf109" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39999 } } peers { permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 } } peers { permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 } }
W20260620 18:45:20.370229 26168 tablet.cc:2369] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:45:20.370142 25976 catalog_manager.cc:5495] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37e1c088eb374185a0f2dec6b48cf109 (127.15.77.66). New cstate: current_term: 1 leader_uuid: "37e1c088eb374185a0f2dec6b48cf109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e1c088eb374185a0f2dec6b48cf109" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 } health_report { overall_health: UNKNOWN } } }
I20260620 18:45:20.384209 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.381622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.15.77.66:39999)
0620 18:45:20.381672 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 18:45:20.381675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:45:20.381684 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:45:20.381753 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:45:20.381754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:45:20.381780 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:45:20.381858 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:45:20.381978 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.382488 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.383367 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:45:20.383569 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:45:20.383651 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:45:20.383838 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:45:20.383946 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:45:20.384045 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 18:45:20.384064 (+ 19us) server_negotiation.cc:299] Negotiation successful
0620 18:45:20.384089 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:45:20.386014 26389 raft_consensus.cc:1261] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499 [term 1 FOLLOWER]: Refusing update from remote peer 37e1c088eb374185a0f2dec6b48cf109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:45:20.386022 26121 raft_consensus.cc:1261] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a [term 1 FOLLOWER]: Refusing update from remote peer 37e1c088eb374185a0f2dec6b48cf109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:45:20.386377 26459 consensus_queue.cc:1035] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c02e9c5c52a942148af6ca4f9935f499" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 32799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:45:20.386508 26462 consensus_queue.cc:1035] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e02ced161817401a98a95608351a687a" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 38051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:45:20.389025 26467 mvcc.cc:205] Tried to move back new op lower bound from 7298994669098434560 to 7298994669032812544. Current Snapshot: MvccSnapshot[applied={T|T < 7298994669098434560}]
I20260620 18:45:20.389344 26466 mvcc.cc:205] Tried to move back new op lower bound from 7298994669098434560 to 7298994669032812544. Current Snapshot: MvccSnapshot[applied={T|T < 7298994669098434560 or (T in {7298994669098434560})}]
I20260620 18:45:20.391288 25976 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:45:20.391439 25976 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:45:20.392190 25976 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee206ab6a6fd48539a302d6599d6afd4: Sending DeleteTablet for 3 replicas of tablet c01784f345a24281b618fe067c036cc4
I20260620 18:45:20.392508 26369 tablet_service.cc:1514] Processing DeleteTablet for tablet c01784f345a24281b618fe067c036cc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:45:20 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:52828
I20260620 18:45:20.392611 26101 tablet_service.cc:1514] Processing DeleteTablet for tablet c01784f345a24281b618fe067c036cc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:45:20 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:47368
I20260620 18:45:20.392712 26235 tablet_service.cc:1514] Processing DeleteTablet for tablet c01784f345a24281b618fe067c036cc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:45:20 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:36268
I20260620 18:45:20.392874 26476 tablet_replica.cc:330] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a: stopping tablet replica
I20260620 18:45:20.392956 26477 tablet_replica.cc:330] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: stopping tablet replica
I20260620 18:45:20.393028 26476 raft_consensus.cc:2229] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:45:20.393035 26475 tablet_replica.cc:330] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: stopping tablet replica
I20260620 18:45:20.393075 26477 raft_consensus.cc:2229] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:45:20.393136 26475 raft_consensus.cc:2229] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:45:20.393203 26477 raft_consensus.cc:2258] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:45:20.393224 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 26038
I20260620 18:45:20.393234 26476 raft_consensus.cc:2258] T c01784f345a24281b618fe067c036cc4 P e02ced161817401a98a95608351a687a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:45:20.393317 26475 raft_consensus.cc:2258] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:45:20.394245 26477 ts_tablet_manager.cc:1905] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:45:20.394407 26475 ts_tablet_manager.cc:1905] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:45:20.395608 26477 ts_tablet_manager.cc:1918] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:45:20.395668 26477 log.cc:1196] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/c01784f345a24281b618fe067c036cc4
I20260620 18:45:20.395864 26477 ts_tablet_manager.cc:1939] T c01784f345a24281b618fe067c036cc4 P 37e1c088eb374185a0f2dec6b48cf109: Deleting consensus metadata
I20260620 18:45:20.396317 25960 catalog_manager.cc:4841] TS 37e1c088eb374185a0f2dec6b48cf109 (127.15.77.66:39999): tablet c01784f345a24281b618fe067c036cc4 (table test-table [id=97d3ea65be294f0c88f89b2c9a3f20db]) successfully deleted
I20260620 18:45:20.396534 26475 ts_tablet_manager.cc:1918] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:45:20.396613 26475 log.cc:1196] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/c01784f345a24281b618fe067c036cc4
I20260620 18:45:20.396881 26475 ts_tablet_manager.cc:1939] T c01784f345a24281b618fe067c036cc4 P c02e9c5c52a942148af6ca4f9935f499: Deleting consensus metadata
I20260620 18:45:20.397353 25963 catalog_manager.cc:4841] TS c02e9c5c52a942148af6ca4f9935f499 (127.15.77.67:32799): tablet c01784f345a24281b618fe067c036cc4 (table test-table [id=97d3ea65be294f0c88f89b2c9a3f20db]) successfully deleted
W20260620 18:45:20.401041 25960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:38051 (error 108)
I20260620 18:45:20.401168 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 26172
I20260620 18:45:20.403846 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:45:20.403637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:38051 (local address 127.0.0.1:47382)
0620 18:45:20.403758 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 18:45:20.403811 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:38051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260620 18:45:20.404021 25960 catalog_manager.cc:4584] TS e02ced161817401a98a95608351a687a (127.15.77.65:38051): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c01784f345a24281b618fe067c036cc4: Network error: Client connection negotiation failed: client connection to 127.15.77.65:38051: connect: Connection refused (error 111)
I20260620 18:45:20.405886 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 26306
I20260620 18:45:20.410377 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 25944
2026-06-20T18:45:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2029 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:45:20 dist-test-slave-tvl9 krb5kdc[26481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:45:20 dist-test-slave-tvl9 krb5kdc[26481](info): set up 2 sockets
Jun 20 18:45:20 dist-test-slave-tvl9 krb5kdc[26481](info): commencing operation
krb5kdc: starting...
W20260620 18:45:21.455598 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:45:21 dist-test-slave-tvl9 krb5kdc[26481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:45:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:45:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.77.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.77.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/rangeradmin_127.15.77.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/rangeradmin_127.15.77.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.77.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.77.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/rangerlookup_127.15.77.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/rangerlookup_127.15.77.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.77.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.77.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/HTTP_127.15.77.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/HTTP_127.15.77.84@KRBTEST.COM.keytab.
I20260620 18:45:21.622355 15669 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task13qjC6/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/postgres -l logfile start
2026-06-20 18:45:24.129 UTC [26511] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 18:45:24.129 UTC [26511] LOG: listening on IPv4 address "127.15.77.84", port 59287
2026-06-20 18:45:24.133 UTC [26511] LOG: listening on Unix socket "/tmp/.s.PGSQL.59287"
2026-06-20 18:45:24.154 UTC [26514] LOG: database system was shut down at 2026-06-20 18:45:22 UTC
2026-06-20 18:45:24.158 UTC [26511] LOG: database system is ready to accept connections
I20260620 18:45:25.111450 15669 mini_postgres.cc:96] Postgres bound to 59287
2026-06-20 18:45:25.117 UTC [26522] FATAL: database "slave" does not exist
127.15.77.84:59287 - accepting connections
I20260620 18:45:25.118067 15669 mini_ranger.cc:161] Starting Ranger...
I20260620 18:45:25.132269 15669 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 18:45:25.596541 15669 mini_ranger.cc:87] Created ranger Postgres database
I20260620 18:45:25.596625 15669 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-admin
2026-06-20 18:45:25,898 [I] DB FLAVOR :POSTGRES
2026-06-20 18:45:25,898 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 18:45:25,898 [I] Checking connection..
2026-06-20 18:45:25,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 18:45:26,175 [I] Checking connection passed.
2026-06-20 18:45:26,175 [I] --------- Verifying version history table ---------
2026-06-20 18:45:26,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 18:45:26,433 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 18:45:26,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 18:45:26,669 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 18:45:26,669 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 18:45:26,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 18:45:26.899 UTC [26586] WARNING: there is no transaction in progress
2026-06-20 18:45:26,905 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 18:45:26,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 18:45:27,148 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 18:45:27,149 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 18:45:27,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 18:45:27,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 18:45:27,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tvl9.c.gcp-upstream.internal','N') ;"
2026-06-20 18:45:27,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 18:45:28,084 [I] Table x_portal_user does not exist in database ranger
2026-06-20 18:45:28,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 18:45:28,330 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 18:45:28,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 18:45:28,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 18:45:28,798 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 18:45:28,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 18:45:29.771 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.787 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.803 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.818 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.833 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.851 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.915 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.921 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.936 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.946 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.957 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.970 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.980 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.986 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:29.993 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:30.000 UTC [26694] WARNING: there is no transaction in progress
2026-06-20 18:45:30,640 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 18:45:30,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 18:45:30,879 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 18:45:30,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 18:45:31,148 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 18:45:31,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 18:45:31,423 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 18:45:31,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 18:45:31,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 18:45:31,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tvl9.c.gcp-upstream.internal';"
2026-06-20 18:45:32,141 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 18:45:32,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 18:45:32,399 [I] DB_PATCHES have already been applied
I20260620 18:45:32.405107 15669 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-admin:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task13qjC6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 18:45:32.405176 15669 mini_ranger.cc:193] Using host: 127.15.77.84
I20260620 18:45:32.408318 15669 mini_ranger.cc:227] Ranger admin URL: http://127.15.77.84:44035
Jun 20, 2026 6:45:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 6:45:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 6:45:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 6:45:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 6:45:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 6:45:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 6:45:33 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 18:45:33.500643 15669 mini_ranger.cc:160] Time spent starting Ranger: real 8.383s user 0.000s sys 0.009s
Jun 20, 2026 6:45:53 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 6:45:53 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 6:45:53 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 6:45:53 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 6:45:54 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 18:45:54.774703 15669 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.77.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.77.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/rangerkms_127.15.77.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/rangerkms_127.15.77.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.77.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.77.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.77.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/HTTP_127.15.77.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.77.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/HTTP_127.15.77.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 18:45:54 dist-test-slave-tvl9 krb5kdc[26481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981154, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 18:45:54.858721 15669 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 18:45:54.868510 15669 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 18:45:55.745864 15669 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 18:45:55.745954 15669 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-kms
I20260620 18:45:55.745973 15669 mini_ranger_kms.cc:227] Using postgres at 127.15.77.84:59287
2026-06-20 18:45:56,015 [I] DB FLAVOR :POSTGRES
2026-06-20 18:45:56,016 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 18:45:56,016 [I] Checking connection
2026-06-20 18:45:56,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 18:45:56,286 [I] connection success
2026-06-20 18:45:56,286 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 18:45:56,286 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 18:45:56,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 18:45:56,516 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 18:45:56,516 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 18:45:56,516 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 18:45:56,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:59287/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 18:45:56,767 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 18:45:57.246091 15669 mini_ranger_kms.cc:310] Created kms service
I20260620 18:45:57.444434 15669 mini_ranger_kms.cc:326] Created kudu user
I20260620 18:45:57.488268 15669 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 18:45:57 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981154, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:45:57.872893 15669 mini_ranger_kms.cc:384] Added ranger policy
I20260620 18:45:57.873008 15669 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-kms:/tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 18:45:57.873029 15669 mini_ranger_kms.cc:240] Using host: 127.15.77.84
I20260620 18:45:57.876734 15669 mini_ranger_kms.cc:276] Ranger KMS PID: 26874
I20260620 18:45:57.876853 15669 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.77.84:53093
Jun 20, 2026 6:45:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 6:45:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 6:45:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 6:45:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 6:45:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 6:45:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 18:46:00.133739 15669 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.275s user 0.004s sys 0.167s
I20260620 18:46:00.133857 15669 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 18:46:00.133947 15669 mini_ranger_kms.cc:401] 127.15.77.84:53093/kms/v1/keys
Jun 20 18:46:00 dist-test-slave-tvl9 krb5kdc[26481](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781981160, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.77.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 18:46:00 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981160, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.77.84@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
Jun 20 18:46:01 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981160, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.77.84@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
Jun 20 18:46:03 dist-test-slave-tvl9 krb5kdc[26481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:46:03.278573 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34689
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:34689
--ranger_config_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:53093/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:03.363463 26925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:03.363677 26925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:03.363699 26925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:03.364915 26925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:46:03.364964 26925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:03.364979 26925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:03.364990 26925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:46:03.365002 26925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:46:03.366628 26925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.77.84:53093/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:34689
--ranger_config_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:03.366899 26925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:03.367174 26925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:03.372504 26930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:03.372532 26925 server_base.cc:1034] running on GCE node
W20260620 18:46:03.372507 26931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:03.372594 26933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:03.373077 26925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:03.373418 26925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:03.374625 26925 hybrid_clock.cc:648] HybridClock initialized: now 1781981163374597 us; error 43 us; skew 500 ppm
Jun 20 18:46:03 dist-test-slave-tvl9 krb5kdc[26481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981163, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:03.377434 26925 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:46:03.377969 26925 webserver.cc:460] Webserver started at http://127.15.77.126:43731/ using document root <none> and password file <none>
I20260620 18:46:03.378221 26925 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:03.378283 26925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:03.378440 26925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 18:46:03 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981163, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:03.424933 26925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "6f2d2fedd33a40178cf5805dfd60d864"
format_stamp: "Formatted at 2026-06-20 18:46:03 on dist-test-slave-tvl9"
server_key: "4462496061039ed10bfc67282c5e145b"
server_key_iv: "3e26e17f265dd2ebdfd5a97c25508b74"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:03.425375 26925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "6f2d2fedd33a40178cf5805dfd60d864"
format_stamp: "Formatted at 2026-06-20 18:46:03 on dist-test-slave-tvl9"
server_key: "4462496061039ed10bfc67282c5e145b"
server_key_iv: "3e26e17f265dd2ebdfd5a97c25508b74"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:03.426827 26925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:46:03.439308 26940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:03.439625 26925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:46:03.439724 26925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "6f2d2fedd33a40178cf5805dfd60d864"
format_stamp: "Formatted at 2026-06-20 18:46:03 on dist-test-slave-tvl9"
server_key: "4462496061039ed10bfc67282c5e145b"
server_key_iv: "3e26e17f265dd2ebdfd5a97c25508b74"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:03.439837 26925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:03.458698 26925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:03.461145 26925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:03.461351 26925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:03.466792 26925 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:34689
I20260620 18:46:03.466816 26992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:34689 every 8 connection(s)
I20260620 18:46:03.467260 26925 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:46:03.468171 26993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:03.470664 26993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864: Bootstrap starting.
I20260620 18:46:03.471365 26993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:03.471716 26993 log.cc:824] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:03.472473 26993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864: No bootstrap required, opened a new log
I20260620 18:46:03.474792 26993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2d2fedd33a40178cf5805dfd60d864" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34689 } }
I20260620 18:46:03.474998 26993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:03.475004 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 26925
I20260620 18:46:03.475073 26993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2d2fedd33a40178cf5805dfd60d864, State: Initialized, Role: FOLLOWER
I20260620 18:46:03.475102 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:46:03.475279 15669 external_mini_cluster.cc:1428] Setting key 6e48634a4b29b4fb21d64d0206743e71
I20260620 18:46:03.475278 26993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2d2fedd33a40178cf5805dfd60d864" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34689 } }
I20260620 18:46:03.475419 26993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:46:03.475514 26993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:46:03.475603 26993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:03.476491 26993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2d2fedd33a40178cf5805dfd60d864" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34689 } }
I20260620 18:46:03.476625 26993 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f2d2fedd33a40178cf5805dfd60d864; no voters:
I20260620 18:46:03.476816 26993 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:46:03.476867 26998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:46:03.477111 26998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [term 1 LEADER]: Becoming Leader. State: Replica: 6f2d2fedd33a40178cf5805dfd60d864, State: Running, Role: LEADER
I20260620 18:46:03.477234 26993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:46:03.477252 26998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2d2fedd33a40178cf5805dfd60d864" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34689 } }
I20260620 18:46:03.477659 26998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2d2fedd33a40178cf5805dfd60d864. Latest consensus state: current_term: 1 leader_uuid: "6f2d2fedd33a40178cf5805dfd60d864" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2d2fedd33a40178cf5805dfd60d864" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34689 } } }
I20260620 18:46:03.477733 26998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:03.477753 27000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f2d2fedd33a40178cf5805dfd60d864" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2d2fedd33a40178cf5805dfd60d864" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34689 } } }
I20260620 18:46:03.477808 27000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864 [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:03.479562 26993 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 18:46:03.479998 27002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:46:03.483246 27002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 18:46:03 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:46:03.487759 27002 catalog_manager.cc:1347] Generated new cluster ID: 43c7375866ef4da5a2f781eb1b245392
I20260620 18:46:03.487828 27002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:46:03.492786 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:03.475814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.15.77.126:34689)
0620 18:46:03.475991 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:03.476001 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:03.476037 (+ 36us) server_negotiation.cc:407] Connection header received
0620 18:46:03.476172 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:03.476180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:03.476237 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:03.476391 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:03.477307 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:03.478992 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:03.481603 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:03.483267 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:03.483490 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:03.487304 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:03.487337 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:03.487337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:03.487357 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:03.490714 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:03.491162 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:03.491166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:03.491167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:03.491284 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:03.491537 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:03.491538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:03.491538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:03.491885 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:03.492033 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:03.492300 (+ 267us) server_negotiation.cc:299] Negotiation successful
0620 18:46:03.492483 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260620 18:46:03.498220 27002 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:46:03.499197 27002 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:46:03.508563 27002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864: Generated new TSK 0
I20260620 18:46:03.508885 27002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:46:04.290702 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:53093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34689
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:04.402750 27038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:04.403052 27038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:04.403093 27038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:04.404939 27038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:04.405000 27038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:04.405040 27038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:46:04.407256 27038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.77.84:53093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:04.407687 27038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:04.407968 27038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:04.414526 27044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:04.414669 27038 server_base.cc:1034] running on GCE node
W20260620 18:46:04.414722 27043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:04.414793 27046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:04.415154 27038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:04.415524 27038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:04.416725 27038 hybrid_clock.cc:648] HybridClock initialized: now 1781981164416693 us; error 46 us; skew 500 ppm
Jun 20 18:46:04 dist-test-slave-tvl9 krb5kdc[26481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:04.423180 27038 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:46:04.423810 27038 webserver.cc:460] Webserver started at http://127.15.77.65:35159/ using document root <none> and password file <none>
I20260620 18:46:04.424082 27038 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:04.424152 27038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:04.424322 27038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 18:46:04 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:04.437743 27038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "0714fd91219248e19de4454b3badb41d"
format_stamp: "Formatted at 2026-06-20 18:46:04 on dist-test-slave-tvl9"
server_key: "e51e05bfc568eb7de70dcb9ecf2b23c7"
server_key_iv: "d8c6198c40eff9473813da8abf199e19"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:04.442879 27038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "0714fd91219248e19de4454b3badb41d"
format_stamp: "Formatted at 2026-06-20 18:46:04 on dist-test-slave-tvl9"
server_key: "e51e05bfc568eb7de70dcb9ecf2b23c7"
server_key_iv: "d8c6198c40eff9473813da8abf199e19"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:04.445102 27038 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260620 18:46:04.461062 27055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:04.461270 27038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260620 18:46:04.461369 27038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "0714fd91219248e19de4454b3badb41d"
format_stamp: "Formatted at 2026-06-20 18:46:04 on dist-test-slave-tvl9"
server_key: "e51e05bfc568eb7de70dcb9ecf2b23c7"
server_key_iv: "d8c6198c40eff9473813da8abf199e19"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:04.461534 27038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:04.479664 27038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:04.481837 27038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:04.482005 27038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:04.482266 27038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:46:04.482679 27038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:46:04.482739 27038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:04.482772 27038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:46:04.482789 27038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:04.490216 27038 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:39009
I20260620 18:46:04.490872 27038 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:46:04.491448 27168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:39009 every 8 connection(s)
I20260620 18:46:04.498190 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 27038
I20260620 18:46:04.498303 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:46:04.498519 15669 external_mini_cluster.cc:1428] Setting key cf342f95ef42c157cd27e1b4e50109ed
Jun 20 18:46:04 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:46:04.506465 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:04.492060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:44977 (local address 127.15.77.126:34689)
0620 18:46:04.492242 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:04.492244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:04.495675 (+ 3431us) server_negotiation.cc:407] Connection header received
0620 18:46:04.496386 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:04.496387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:04.496404 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:04.496502 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:04.497286 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:04.498084 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:04.499624 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:04.499933 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:04.500020 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:04.502743 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:04.502745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:04.502745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:04.502753 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:04.504716 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:04.505270 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:04.505272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:04.505272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:04.505341 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:04.505659 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:04.505660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:04.505661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:04.505819 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:04.505899 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:04.506244 (+ 345us) server_negotiation.cc:299] Negotiation successful
0620 18:46:04.506344 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260620 18:46:04.506798 27171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:04.491730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:34689 (local address 127.15.77.65:44977)
0620 18:46:04.495482 (+ 3752us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:04.495543 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:04.496253 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:04.496599 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:04.496604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:04.496883 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:04.497175 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:04.497182 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:04.498198 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:04.498200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:04.499527 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:04.499530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:04.500036 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:04.500037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:04.500194 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:04.500855 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:04.500867 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:04.502632 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:04.504808 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:04.504815 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:04.504819 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:04.505179 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:04.505431 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:04.505432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:04.505432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:04.505583 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:04.505904 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:04.505910 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:04.506079 (+ 169us) client_negotiation.cc:769] Sending connection context
0620 18:46:04.506207 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 18:46:04.506365 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3678,"thread_start_us":83,"threads_started":1}
I20260620 18:46:04.507345 27169 heartbeater.cc:347] Connected to a master server at 127.15.77.126:34689
I20260620 18:46:04.507468 27169 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:04.507683 27169 heartbeater.cc:511] Master 127.15.77.126:34689 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:46:04.508296 26957 ts_manager.cc:194] Registered new tserver with Master: 0714fd91219248e19de4454b3badb41d (127.15.77.65:39009)
I20260620 18:46:04.509045 26957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:44977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:46:04.530560 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:53093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34689
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:04.631227 27175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:04.631513 27175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:04.631552 27175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:04.632833 27175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:04.632905 27175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:04.632949 27175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:46:04.634482 27175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.77.84:53093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:04.634750 27175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:04.634979 27175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Jun 20 18:46:04 dist-test-slave-tvl9 krb5kdc[26481](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 18:46:04.640257 27183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:04.640260 27185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:04.640564 27182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:04.641386 27175 server_base.cc:1034] running on GCE node
I20260620 18:46:04.641601 27175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:04.641887 27175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:04.643115 27175 hybrid_clock.cc:648] HybridClock initialized: now 1781981164643089 us; error 37 us; skew 500 ppm
Jun 20 18:46:04 dist-test-slave-tvl9 krb5kdc[26481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:04.646001 27175 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:46:04.646590 27175 webserver.cc:460] Webserver started at http://127.15.77.66:41235/ using document root <none> and password file <none>
I20260620 18:46:04.646890 27175 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:04.646957 27175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:04.647106 27175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 18:46:04 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:04.659852 27175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "48009e49709c447da7102d9cda678364"
format_stamp: "Formatted at 2026-06-20 18:46:04 on dist-test-slave-tvl9"
server_key: "ec4d80324c3d1a5a0c3fa6b3e1a9846f"
server_key_iv: "096c585ab01a208901b998b9b4ce827e"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:04.660391 27175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "48009e49709c447da7102d9cda678364"
format_stamp: "Formatted at 2026-06-20 18:46:04 on dist-test-slave-tvl9"
server_key: "ec4d80324c3d1a5a0c3fa6b3e1a9846f"
server_key_iv: "096c585ab01a208901b998b9b4ce827e"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:04.668545 27175 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.000s
I20260620 18:46:04.685513 27192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:04.685796 27175 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260620 18:46:04.685930 27175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "48009e49709c447da7102d9cda678364"
format_stamp: "Formatted at 2026-06-20 18:46:04 on dist-test-slave-tvl9"
server_key: "ec4d80324c3d1a5a0c3fa6b3e1a9846f"
server_key_iv: "096c585ab01a208901b998b9b4ce827e"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:04.686086 27175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:04.722748 27175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:04.728717 27175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:04.728947 27175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:04.729388 27175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:46:04.729828 27175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:46:04.729923 27175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:04.729995 27175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:46:04.730065 27175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:04.736486 27175 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:34933
I20260620 18:46:04.737005 27175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:46:04.737972 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 27175
I20260620 18:46:04.738085 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:46:04.738272 15669 external_mini_cluster.cc:1428] Setting key c667aa186617307026158c99cb83ae45
I20260620 18:46:04.743904 27306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:34933 every 8 connection(s)
Jun 20 18:46:04 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
I20260620 18:46:04.756390 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:04.745047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:59517 (local address 127.15.77.126:34689)
0620 18:46:04.745099 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:04.745101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:04.745438 (+ 337us) server_negotiation.cc:407] Connection header received
0620 18:46:04.746248 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:04.746250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:04.746267 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:04.746299 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:04.747155 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:04.747937 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:04.749528 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:04.749842 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:04.749890 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:04.752639 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:04.752641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:04.752641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:04.752649 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:04.754356 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:04.754905 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:04.754906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:04.754906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:04.754966 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:04.755560 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:04.755562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:04.755562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:04.755711 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:04.755755 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:04.756197 (+ 442us) server_negotiation.cc:299] Negotiation successful
0620 18:46:04.756290 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:46:04.757153 27310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:04.744746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:34689 (local address 127.15.77.66:59517)
0620 18:46:04.745301 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:04.745352 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:04.746110 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:04.746489 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:04.746493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:04.746739 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:04.747042 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:04.747048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:04.748071 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:04.748072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:04.749423 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:04.749427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:04.749984 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:04.749985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:04.750137 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:04.750768 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:04.750778 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:04.752525 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:04.754457 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:04.754463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:04.754466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:04.754756 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:04.755293 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:04.755294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:04.755294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:04.755458 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:04.755804 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:04.755811 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:04.756033 (+ 222us) client_negotiation.cc:769] Sending connection context
0620 18:46:04.756720 (+ 687us) client_negotiation.cc:240] Negotiation successful
0620 18:46:04.756896 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":96,"threads_started":1}
I20260620 18:46:04.757725 27307 heartbeater.cc:347] Connected to a master server at 127.15.77.126:34689
I20260620 18:46:04.757861 27307 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:04.758077 27307 heartbeater.cc:511] Master 127.15.77.126:34689 requested a full tablet report, sending...
I20260620 18:46:04.758575 26957 ts_manager.cc:194] Registered new tserver with Master: 48009e49709c447da7102d9cda678364 (127.15.77.66:34933)
I20260620 18:46:04.759079 26957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:59517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:46:04.781997 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:53093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34689
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:04.909139 27312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:04.909472 27312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:04.909551 27312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:04.911509 27312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:04.911584 27312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:04.911629 27312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:46:04.914074 27312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.77.84:53093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:04.914368 27312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:04.914674 27312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:04.921734 27317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:04.921773 27318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:04.922008 27320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:04.923081 27312 server_base.cc:1034] running on GCE node
I20260620 18:46:04.923345 27312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:04.923656 27312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:04.939251 27312 hybrid_clock.cc:648] HybridClock initialized: now 1781981164939199 us; error 62 us; skew 500 ppm
Jun 20 18:46:04 dist-test-slave-tvl9 krb5kdc[26481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:04.943137 27312 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:46:04.944093 27312 webserver.cc:460] Webserver started at http://127.15.77.67:46693/ using document root <none> and password file <none>
I20260620 18:46:04.944432 27312 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:04.944677 27312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:04.944938 27312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 18:46:04 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:04.964018 27312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "bf295f97bbc1429d8fb49b9a7e00014a"
format_stamp: "Formatted at 2026-06-20 18:46:04 on dist-test-slave-tvl9"
server_key: "96aa31fb0746b9850ddce813100a6857"
server_key_iv: "b86dc6a532ef1ddaee8e39ebdcd5afbd"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:04.964486 27312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "bf295f97bbc1429d8fb49b9a7e00014a"
format_stamp: "Formatted at 2026-06-20 18:46:04 on dist-test-slave-tvl9"
server_key: "96aa31fb0746b9850ddce813100a6857"
server_key_iv: "b86dc6a532ef1ddaee8e39ebdcd5afbd"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:04.966465 27312 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260620 18:46:04.982477 27327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:04.983568 27312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 18:46:04.983700 27312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "bf295f97bbc1429d8fb49b9a7e00014a"
format_stamp: "Formatted at 2026-06-20 18:46:04 on dist-test-slave-tvl9"
server_key: "96aa31fb0746b9850ddce813100a6857"
server_key_iv: "b86dc6a532ef1ddaee8e39ebdcd5afbd"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:04.983839 27312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:04.994715 27312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:04.997251 27312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:04.997411 27312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:04.997718 27312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:46:04.998126 27312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:46:04.998170 27312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:04.998199 27312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:46:04.998216 27312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:05.004793 27312 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:43919
I20260620 18:46:05.005261 27312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:46:05.013350 27440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:43919 every 8 connection(s)
I20260620 18:46:05.020143 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 27312
I20260620 18:46:05.020247 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:46:05.020429 15669 external_mini_cluster.cc:1428] Setting key bc801bd12d6c93af27f6c2393a20427d
Jun 20 18:46:05 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:46:05.038681 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.014184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:60665 (local address 127.15.77.126:34689)
0620 18:46:05.014233 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:05.014236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:05.019667 (+ 5431us) server_negotiation.cc:407] Connection header received
0620 18:46:05.021896 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:05.021898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:05.021914 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:05.021950 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:05.022889 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.023708 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.025281 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.025588 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.025650 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.030301 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:05.030303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:05.030304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:05.030313 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:05.032845 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:05.037388 (+ 4543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:05.037390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:05.037390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:05.037466 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:05.037818 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:05.037820 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:05.037820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:05.037973 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:05.038036 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:05.038462 (+ 426us) server_negotiation.cc:299] Negotiation successful
0620 18:46:05.038570 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:46:05.039613 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.014185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:34689 (local address 127.15.77.67:60665)
0620 18:46:05.019464 (+ 5279us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:05.019536 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:05.020286 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:05.022177 (+ 1891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:05.022182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:05.022455 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:05.022770 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.022776 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.023847 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.023848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.025171 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.025175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.025757 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.025758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.025907 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.026552 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:05.026563 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:05.029987 (+ 3424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:05.036786 (+ 6799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:05.036800 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:05.036804 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:05.037235 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:05.037547 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:05.037548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:05.037549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:05.037728 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:05.038076 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:05.038083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:05.038274 (+ 191us) client_negotiation.cc:769] Sending connection context
0620 18:46:05.039101 (+ 827us) client_negotiation.cc:240] Negotiation successful
0620 18:46:05.039288 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5193,"thread_start_us":90,"threads_started":1}
I20260620 18:46:05.040315 27441 heartbeater.cc:347] Connected to a master server at 127.15.77.126:34689
I20260620 18:46:05.040413 27441 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:05.040632 27441 heartbeater.cc:511] Master 127.15.77.126:34689 requested a full tablet report, sending...
I20260620 18:46:05.041077 26957 ts_manager.cc:194] Registered new tserver with Master: bf295f97bbc1429d8fb49b9a7e00014a (127.15.77.67:43919)
I20260620 18:46:05.041661 26957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:60665
I20260620 18:46:05.043220 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:46:05.066556 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.051669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.15.77.126:34689)
0620 18:46:05.051729 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:05.051731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:05.055592 (+ 3861us) server_negotiation.cc:407] Connection header received
0620 18:46:05.055733 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:05.055735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:05.055749 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:05.055782 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:05.056666 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.057418 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.058528 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.058828 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.058880 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.061712 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:05.061714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:05.061714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:05.061723 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:05.063543 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:05.065603 (+ 2060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:05.065605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:05.065606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:05.065668 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:05.065886 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:05.065887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:05.065887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:05.066025 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:05.066120 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:05.066365 (+ 245us) server_negotiation.cc:299] Negotiation successful
0620 18:46:05.066450 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 18:46:05.068889 26957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:46:05.071342 26957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:46:05.082064 27459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.075066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:43919 (local address 127.0.0.1:43908)
0620 18:46:05.076334 (+ 1268us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:05.076380 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:05.076448 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:05.076977 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:05.076981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:05.076984 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:46:05.077057 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.077059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.078884 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.078885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.080632 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.080635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.081731 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.081732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.081816 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.081831 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 18:46:05.081902 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 18:46:05.081962 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"thread_start_us":96,"threads_started":1}
I20260620 18:46:05.082734 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.075464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43908 (local address 127.15.77.67:43919)
0620 18:46:05.076723 (+ 1259us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:05.076729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:05.076739 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:46:05.076781 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:05.076783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:05.076800 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:05.076878 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:46:05.077297 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.078781 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.080748 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.081533 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.081609 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.081693 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:05.082603 (+ 910us) server_negotiation.cc:299] Negotiation successful
0620 18:46:05.082652 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1217,"thread_start_us":78,"threads_started":1}
I20260620 18:46:05.083762 27375 tablet_service.cc:1467] Processing CreateTablet for tablet 64cf916415204ff9b0096817d2ca8d3f (DEFAULT_TABLE table=test-table [id=b9009af9c8284d26aade14444d21066b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:46:05.084172 27375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64cf916415204ff9b0096817d2ca8d3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:05.088739 27460 tablet_bootstrap.cc:492] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: Bootstrap starting.
I20260620 18:46:05.089931 27460 tablet_bootstrap.cc:654] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:05.090377 27460 log.cc:824] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:05.091264 27460 tablet_bootstrap.cc:492] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: No bootstrap required, opened a new log
I20260620 18:46:05.091809 27460 ts_tablet_manager.cc:1397] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 18:46:05.094010 27460 raft_consensus.cc:348] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 } } peers { permanent_uuid: "48009e49709c447da7102d9cda678364" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 34933 } } peers { permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 } }
I20260620 18:46:05.094211 27460 raft_consensus.cc:374] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:05.094259 27460 raft_consensus.cc:729] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf295f97bbc1429d8fb49b9a7e00014a, State: Initialized, Role: FOLLOWER
I20260620 18:46:05.094589 27460 consensus_queue.cc:260] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 } } peers { permanent_uuid: "48009e49709c447da7102d9cda678364" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 34933 } } peers { permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 } }
I20260620 18:46:05.095007 27460 ts_tablet_manager.cc:1428] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:46:05.095198 27441 heartbeater.cc:503] Master 127.15.77.126:34689 was elected leader, sending a full tablet report...
I20260620 18:46:05.095508 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.075977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:39009 (local address 127.0.0.1:38982)
0620 18:46:05.077179 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:05.077188 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:05.077227 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:05.087737 (+ 10510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:05.087738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:05.087742 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:46:05.087799 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.087801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.090746 (+ 2945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.090748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.091945 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.091949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.095236 (+ 3287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.095237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.095305 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.095333 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 18:46:05.095369 (+ 36us) client_negotiation.cc:240] Negotiation successful
0620 18:46:05.095417 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1184,"thread_start_us":60,"threads_started":1}
I20260620 18:46:05.096160 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.075476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38982 (local address 127.15.77.65:39009)
0620 18:46:05.087434 (+ 11958us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:05.087441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:05.087458 (+ 17us) server_negotiation.cc:407] Connection header received
0620 18:46:05.087500 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:05.087503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:05.087522 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:05.087633 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:46:05.087886 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.088974 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.092061 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.092887 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.095883 (+ 2996us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.095975 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:05.096006 (+ 31us) server_negotiation.cc:299] Negotiation successful
0620 18:46:05.096060 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11909,"thread_start_us":59,"threads_started":1}
I20260620 18:46:05.097254 27103 tablet_service.cc:1467] Processing CreateTablet for tablet 64cf916415204ff9b0096817d2ca8d3f (DEFAULT_TABLE table=test-table [id=b9009af9c8284d26aade14444d21066b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:46:05.097640 27103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64cf916415204ff9b0096817d2ca8d3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:05.102411 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.075809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:34933 (local address 127.0.0.1:56736)
0620 18:46:05.076554 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:05.076562 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:05.076597 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:05.089366 (+ 12769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:05.089367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:05.089370 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:46:05.089421 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.089423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.093004 (+ 3581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.093005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.094184 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.094187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.102112 (+ 7925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.102113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.102217 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.102224 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 18:46:05.102264 (+ 40us) client_negotiation.cc:240] Negotiation successful
0620 18:46:05.102320 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":84,"threads_started":1}
I20260620 18:46:05.103130 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.075918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.15.77.66:34933)
0620 18:46:05.089172 (+ 13254us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:05.089179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:05.089189 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:46:05.089229 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:05.089231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:05.089249 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:05.089499 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:46:05.089580 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.090622 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.094324 (+ 3702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.095110 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.102818 (+ 7708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.102945 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:05.102978 (+ 33us) server_negotiation.cc:299] Negotiation successful
0620 18:46:05.103030 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13208,"thread_start_us":85,"threads_started":1}
I20260620 18:46:05.103715 27463 tablet_bootstrap.cc:492] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d: Bootstrap starting.
I20260620 18:46:05.104030 27241 tablet_service.cc:1467] Processing CreateTablet for tablet 64cf916415204ff9b0096817d2ca8d3f (DEFAULT_TABLE table=test-table [id=b9009af9c8284d26aade14444d21066b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:46:05.104430 27241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64cf916415204ff9b0096817d2ca8d3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:05.104643 27463 tablet_bootstrap.cc:654] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:05.104990 27463 log.cc:824] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:05.105837 27463 tablet_bootstrap.cc:492] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d: No bootstrap required, opened a new log
I20260620 18:46:05.105907 27463 ts_tablet_manager.cc:1397] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:46:05.108721 27463 raft_consensus.cc:348] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 } } peers { permanent_uuid: "48009e49709c447da7102d9cda678364" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 34933 } } peers { permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 } }
I20260620 18:46:05.109524 27463 raft_consensus.cc:374] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:05.109629 27463 raft_consensus.cc:729] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0714fd91219248e19de4454b3badb41d, State: Initialized, Role: FOLLOWER
I20260620 18:46:05.109844 27463 consensus_queue.cc:260] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 } } peers { permanent_uuid: "48009e49709c447da7102d9cda678364" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 34933 } } peers { permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 } }
I20260620 18:46:05.110258 27463 ts_tablet_manager.cc:1428] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 18:46:05.110464 27169 heartbeater.cc:503] Master 127.15.77.126:34689 was elected leader, sending a full tablet report...
I20260620 18:46:05.110637 27465 tablet_bootstrap.cc:492] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: Bootstrap starting.
I20260620 18:46:05.111977 27465 tablet_bootstrap.cc:654] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:05.112926 27465 log.cc:824] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:05.113813 27465 tablet_bootstrap.cc:492] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: No bootstrap required, opened a new log
I20260620 18:46:05.113888 27465 ts_tablet_manager.cc:1397] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 18:46:05.116076 27465 raft_consensus.cc:348] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 } } peers { permanent_uuid: "48009e49709c447da7102d9cda678364" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 34933 } } peers { permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 } }
I20260620 18:46:05.116273 27465 raft_consensus.cc:374] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:05.116313 27465 raft_consensus.cc:729] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48009e49709c447da7102d9cda678364, State: Initialized, Role: FOLLOWER
I20260620 18:46:05.116432 27465 consensus_queue.cc:260] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 } } peers { permanent_uuid: "48009e49709c447da7102d9cda678364" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 34933 } } peers { permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 } }
I20260620 18:46:05.116708 27465 ts_tablet_manager.cc:1428] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:46:05.116854 27307 heartbeater.cc:503] Master 127.15.77.126:34689 was elected leader, sending a full tablet report...
Jun 20 18:46:05 dist-test-slave-tvl9 krb5kdc[26481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981164, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
W20260620 18:46:05.242254 27170 tablet.cc:2369] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 18:46:05.245611 27308 tablet.cc:2369] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 18:46:05.264735 27442 tablet.cc:2369] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:46:05.293586 27468 raft_consensus.cc:482] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:46:05.293735 27468 raft_consensus.cc:504] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 } } peers { permanent_uuid: "48009e49709c447da7102d9cda678364" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 34933 } } peers { permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 } }
I20260620 18:46:05.294133 27468 leader_election.cc:288] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf295f97bbc1429d8fb49b9a7e00014a (127.15.77.67:43919), 0714fd91219248e19de4454b3badb41d (127.15.77.65:39009)
I20260620 18:46:05.298393 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.294557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:54797 (local address 127.15.77.67:43919)
0620 18:46:05.294599 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:05.294601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:05.294803 (+ 202us) server_negotiation.cc:407] Connection header received
0620 18:46:05.294906 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:05.294907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:05.294917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:05.294946 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:46:05.295151 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.295948 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.297429 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.298007 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.298055 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.298065 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:05.298277 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 18:46:05.298315 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:46:05.298878 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.294344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:43919 (local address 127.15.77.66:54797)
0620 18:46:05.294735 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:05.294747 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:05.294831 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:05.295011 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:05.295011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:05.295014 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:46:05.295085 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.295087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.296071 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.296072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.297322 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.297325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.298126 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.298126 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.298201 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.298205 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:46:05.298743 (+ 538us) client_negotiation.cc:240] Negotiation successful
0620 18:46:05.298792 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":57,"threads_started":1}
I20260620 18:46:05.299342 27376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cf916415204ff9b0096817d2ca8d3f" candidate_uuid: "48009e49709c447da7102d9cda678364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" is_pre_election: true
I20260620 18:46:05.299525 27376 raft_consensus.cc:2454] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48009e49709c447da7102d9cda678364 in term 0.
I20260620 18:46:05.299917 27196 leader_election.cc:302] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48009e49709c447da7102d9cda678364, bf295f97bbc1429d8fb49b9a7e00014a; no voters:
I20260620 18:46:05.300050 27468 raft_consensus.cc:2781] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:46:05.300086 27468 raft_consensus.cc:482] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:46:05.300107 27468 raft_consensus.cc:3037] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:05.300994 27468 raft_consensus.cc:504] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 } } peers { permanent_uuid: "48009e49709c447da7102d9cda678364" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 34933 } } peers { permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 } }
I20260620 18:46:05.301126 27468 leader_election.cc:288] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [CANDIDATE]: Term 1 election: Requested vote from peers bf295f97bbc1429d8fb49b9a7e00014a (127.15.77.67:43919), 0714fd91219248e19de4454b3badb41d (127.15.77.65:39009)
I20260620 18:46:05.301450 27376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cf916415204ff9b0096817d2ca8d3f" candidate_uuid: "48009e49709c447da7102d9cda678364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf295f97bbc1429d8fb49b9a7e00014a"
I20260620 18:46:05.301524 27376 raft_consensus.cc:3037] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:05.302320 27376 raft_consensus.cc:2454] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48009e49709c447da7102d9cda678364 in term 1.
I20260620 18:46:05.302541 27196 leader_election.cc:302] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48009e49709c447da7102d9cda678364, bf295f97bbc1429d8fb49b9a7e00014a; no voters:
I20260620 18:46:05.302652 27468 raft_consensus.cc:2781] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:46:05.302810 27468 raft_consensus.cc:686] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 1 LEADER]: Becoming Leader. State: Replica: 48009e49709c447da7102d9cda678364, State: Running, Role: LEADER
I20260620 18:46:05.302896 27468 consensus_queue.cc:237] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 } } peers { permanent_uuid: "48009e49709c447da7102d9cda678364" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 34933 } } peers { permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 } }
I20260620 18:46:05.303709 26957 catalog_manager.cc:5495] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48009e49709c447da7102d9cda678364 (127.15.77.66). New cstate: current_term: 1 leader_uuid: "48009e49709c447da7102d9cda678364" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48009e49709c447da7102d9cda678364" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 34933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 } health_report { overall_health: UNKNOWN } } }
I20260620 18:46:05.308529 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.294405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:50255 (local address 127.15.77.65:39009)
0620 18:46:05.294456 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:05.294458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:05.304848 (+ 10390us) server_negotiation.cc:407] Connection header received
0620 18:46:05.304898 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:05.304899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:05.304912 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:05.304982 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:46:05.305483 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.306240 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.307566 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.308168 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.308214 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.308224 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:05.308416 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 18:46:05.308455 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:46:05.308871 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.294348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:39009 (local address 127.15.77.66:50255)
0620 18:46:05.304784 (+ 10436us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:05.304796 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:05.304846 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:05.305346 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:05.305347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:05.305349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:46:05.305384 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.305386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.306334 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.306335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.307465 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:05.307467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.308280 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.308280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:05.308343 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.308346 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:46:05.308756 (+ 410us) client_negotiation.cc:240] Negotiation successful
0620 18:46:05.308803 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10421,"thread_start_us":101,"threads_started":1}
I20260620 18:46:05.309306 27122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cf916415204ff9b0096817d2ca8d3f" candidate_uuid: "48009e49709c447da7102d9cda678364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0714fd91219248e19de4454b3badb41d"
I20260620 18:46:05.309330 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cf916415204ff9b0096817d2ca8d3f" candidate_uuid: "48009e49709c447da7102d9cda678364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0714fd91219248e19de4454b3badb41d" is_pre_election: true
I20260620 18:46:05.309448 27122 raft_consensus.cc:3037] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:05.310390 27122 raft_consensus.cc:2454] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48009e49709c447da7102d9cda678364 in term 1.
I20260620 18:46:05.314284 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.310970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56746 (local address 127.15.77.66:34933)
0620 18:46:05.311018 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:05.311020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:05.311030 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:46:05.311135 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:05.311136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:05.311157 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:05.311233 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:46:05.311444 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.312173 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.313291 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:05.313588 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:05.313640 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:05.313829 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:46:05.313958 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:46:05.314026 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:05.314194 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 18:46:05.314212 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:46:05.316613 27122 raft_consensus.cc:1261] T 64cf916415204ff9b0096817d2ca8d3f P 0714fd91219248e19de4454b3badb41d [term 1 FOLLOWER]: Refusing update from remote peer 48009e49709c447da7102d9cda678364: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:46:05.316934 27376 raft_consensus.cc:1261] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a [term 1 FOLLOWER]: Refusing update from remote peer 48009e49709c447da7102d9cda678364: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:46:05.317303 27468 consensus_queue.cc:1035] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0714fd91219248e19de4454b3badb41d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:46:05.317482 27468 consensus_queue.cc:1035] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf295f97bbc1429d8fb49b9a7e00014a" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:46:05.323064 27474 mvcc.cc:205] Tried to move back new op lower bound from 7298994853133414400 to 7298994853081165824. Current Snapshot: MvccSnapshot[applied={T|T < 7298994853133414400}]
I20260620 18:46:05.323745 27475 mvcc.cc:205] Tried to move back new op lower bound from 7298994853133414400 to 7298994853081165824. Current Snapshot: MvccSnapshot[applied={T|T < 7298994853133414400}]
I20260620 18:46:05.325583 26957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:46:05.325685 26957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:46:05.326342 26957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f2d2fedd33a40178cf5805dfd60d864: Sending DeleteTablet for 3 replicas of tablet 64cf916415204ff9b0096817d2ca8d3f
I20260620 18:46:05.326763 27375 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cf916415204ff9b0096817d2ca8d3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:46:05 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:43908
I20260620 18:46:05.326972 27103 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cf916415204ff9b0096817d2ca8d3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:46:05 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:38982
I20260620 18:46:05.327344 27241 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cf916415204ff9b0096817d2ca8d3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:46:05 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:56736
I20260620 18:46:05.327872 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 27038
I20260620 18:46:05.331291 27477 mvcc.cc:205] Tried to move back new op lower bound from 7298994853133414400 to 7298994853081165824. Current Snapshot: MvccSnapshot[applied={T|T < 7298994853133414400}]
I20260620 18:46:05.331617 27485 tablet_replica.cc:330] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: stopping tablet replica
I20260620 18:46:05.331723 27485 raft_consensus.cc:2229] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:46:05.331837 27485 raft_consensus.cc:2258] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:46:05.336333 27485 ts_tablet_manager.cc:1905] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:46:05.337947 27483 tablet_replica.cc:330] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: stopping tablet replica
I20260620 18:46:05.338127 27483 raft_consensus.cc:2229] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:46:05.338212 27483 raft_consensus.cc:2258] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:46:05.338388 27485 ts_tablet_manager.cc:1918] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:46:05.338464 27485 log.cc:1196] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/64cf916415204ff9b0096817d2ca8d3f
I20260620 18:46:05.338747 27485 ts_tablet_manager.cc:1939] T 64cf916415204ff9b0096817d2ca8d3f P 48009e49709c447da7102d9cda678364: Deleting consensus metadata
I20260620 18:46:05.339187 26941 catalog_manager.cc:4841] TS 48009e49709c447da7102d9cda678364 (127.15.77.66:34933): tablet 64cf916415204ff9b0096817d2ca8d3f (table test-table [id=b9009af9c8284d26aade14444d21066b]) successfully deleted
W20260620 18:46:05.342564 26943 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:39009 (error 108)
I20260620 18:46:05.343577 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 27175
I20260620 18:46:05.347707 27483 ts_tablet_manager.cc:1905] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:46:05.349498 27483 ts_tablet_manager.cc:1918] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:46:05.349591 27483 log.cc:1196] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/64cf916415204ff9b0096817d2ca8d3f
I20260620 18:46:05.350242 27483 ts_tablet_manager.cc:1939] T 64cf916415204ff9b0096817d2ca8d3f P bf295f97bbc1429d8fb49b9a7e00014a: Deleting consensus metadata
I20260620 18:46:05.350775 26944 catalog_manager.cc:4841] TS bf295f97bbc1429d8fb49b9a7e00014a (127.15.77.67:43919): tablet 64cf916415204ff9b0096817d2ca8d3f (table test-table [id=b9009af9c8284d26aade14444d21066b]) successfully deleted
I20260620 18:46:05.351751 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:05.351606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:39009 (local address 127.0.0.1:38984)
0620 18:46:05.351662 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:05.351705 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 18:46:05.351881 26943 catalog_manager.cc:4584] TS 0714fd91219248e19de4454b3badb41d (127.15.77.65:39009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64cf916415204ff9b0096817d2ca8d3f: Network error: Client connection negotiation failed: client connection to 127.15.77.65:39009: connect: Connection refused (error 111)
I20260620 18:46:05.353266 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 27312
I20260620 18:46:05.359633 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 26925
I20260620 18:46:05.366760 15669 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 18:46:05.711552 15669 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 18:46:05.711673 15669 mini_ranger.cc:66] Stopping Ranger...
I20260620 18:46:06.056613 15669 mini_ranger.cc:68] Stopped Ranger
2026-06-20 18:46:06.056 UTC [26511] LOG: received smart shutdown request
2026-06-20 18:46:06.060 UTC [26511] LOG: background worker "logical replication launcher" (PID 26520) exited with exit code 1
2026-06-20 18:46:06.060 UTC [26515] LOG: shutting down
2026-06-20 18:46:06.233 UTC [26511] LOG: database system is shut down
2026-06-20T18:46:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45930 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:46:06 dist-test-slave-tvl9 krb5kdc[27496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:46:06 dist-test-slave-tvl9 krb5kdc[27496](info): set up 2 sockets
Jun 20 18:46:06 dist-test-slave-tvl9 krb5kdc[27496](info): commencing operation
krb5kdc: starting...
W20260620 18:46:07.391286 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:46:07 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:46:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:46:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.77.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.77.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/rangeradmin_127.15.77.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/rangeradmin_127.15.77.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.77.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.77.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/rangerlookup_127.15.77.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/rangerlookup_127.15.77.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.77.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.77.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/HTTP_127.15.77.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/HTTP_127.15.77.84@KRBTEST.COM.keytab.
I20260620 18:46:07.557446 15669 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task13qjC6/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/postgres -l logfile start
2026-06-20 18:46:10.054 UTC [27526] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 18:46:10.054 UTC [27526] LOG: listening on IPv4 address "127.15.77.84", port 50867
2026-06-20 18:46:10.058 UTC [27526] LOG: listening on Unix socket "/tmp/.s.PGSQL.50867"
2026-06-20 18:46:10.080 UTC [27529] LOG: database system was shut down at 2026-06-20 18:46:08 UTC
2026-06-20 18:46:10.084 UTC [27526] LOG: database system is ready to accept connections
I20260620 18:46:11.039125 15669 mini_postgres.cc:96] Postgres bound to 50867
2026-06-20 18:46:11.045 UTC [27537] FATAL: database "slave" does not exist
127.15.77.84:50867 - accepting connections
I20260620 18:46:11.045720 15669 mini_ranger.cc:161] Starting Ranger...
I20260620 18:46:11.060457 15669 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 18:46:11.546955 15669 mini_ranger.cc:87] Created ranger Postgres database
I20260620 18:46:11.547046 15669 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-admin
2026-06-20 18:46:11,814 [I] DB FLAVOR :POSTGRES
2026-06-20 18:46:11,815 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 18:46:11,815 [I] Checking connection..
2026-06-20 18:46:11,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 18:46:12,072 [I] Checking connection passed.
2026-06-20 18:46:12,073 [I] --------- Verifying version history table ---------
2026-06-20 18:46:12,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 18:46:12,310 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 18:46:12,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 18:46:12,575 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 18:46:12,575 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 18:46:12,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 18:46:12.834 UTC [27601] WARNING: there is no transaction in progress
2026-06-20 18:46:12,841 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 18:46:12,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 18:46:13,107 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 18:46:13,107 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 18:46:13,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 18:46:13,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 18:46:13,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tvl9.c.gcp-upstream.internal','N') ;"
2026-06-20 18:46:13,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 18:46:14,077 [I] Table x_portal_user does not exist in database ranger
2026-06-20 18:46:14,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 18:46:14,325 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 18:46:14,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 18:46:14,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 18:46:14,826 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 18:46:14,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 18:46:15.764 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.780 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.798 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.813 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.832 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.850 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.907 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.915 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.930 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.941 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.952 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.966 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.976 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.982 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.989 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:15.996 UTC [27709] WARNING: there is no transaction in progress
2026-06-20 18:46:16,682 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 18:46:16,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 18:46:16,925 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 18:46:16,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 18:46:17,180 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 18:46:17,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 18:46:17,422 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 18:46:17,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 18:46:17,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 18:46:17,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tvl9.c.gcp-upstream.internal';"
2026-06-20 18:46:18,138 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 18:46:18,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 18:46:18,400 [I] DB_PATCHES have already been applied
I20260620 18:46:18.405407 15669 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-admin:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task13qjC6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 18:46:18.405493 15669 mini_ranger.cc:193] Using host: 127.15.77.84
I20260620 18:46:18.408689 15669 mini_ranger.cc:227] Ranger admin URL: http://127.15.77.84:37303
Jun 20, 2026 6:46:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 6:46:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 6:46:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 6:46:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 6:46:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 6:46:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 6:46:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 18:46:19.508409 15669 mini_ranger.cc:160] Time spent starting Ranger: real 8.463s user 0.003s sys 0.006s
Jun 20, 2026 6:46:40 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 6:46:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 6:46:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 6:46:40 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 6:46:41 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 18:46:41.666642 15669 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.77.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.77.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/rangerkms_127.15.77.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.77.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/rangerkms_127.15.77.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.77.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.77.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.77.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/HTTP_127.15.77.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.77.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/HTTP_127.15.77.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 18:46:41 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981201, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 18:46:41.769738 15669 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 18:46:41.780357 15669 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 18:46:42.613068 15669 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 18:46:42.613153 15669 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-kms
I20260620 18:46:42.613175 15669 mini_ranger_kms.cc:227] Using postgres at 127.15.77.84:50867
2026-06-20 18:46:42,780 [I] DB FLAVOR :POSTGRES
2026-06-20 18:46:42,780 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 18:46:42,780 [I] Checking connection
2026-06-20 18:46:42,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 18:46:43,041 [I] connection success
2026-06-20 18:46:43,041 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 18:46:43,041 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 18:46:43,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 18:46:43,289 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 18:46:43,289 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 18:46:43,289 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 18:46:43,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.77.84:50867/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 18:46:43,557 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 18:46:44.043046 15669 mini_ranger_kms.cc:310] Created kms service
I20260620 18:46:44.261603 15669 mini_ranger_kms.cc:326] Created kudu user
I20260620 18:46:44.306937 15669 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 18:46:44 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981201, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:44.722640 15669 mini_ranger_kms.cc:384] Added ranger policy
I20260620 18:46:44.722888 15669 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-kms:/tmp/dist-test-task13qjC6/build/release/bin/postgresql.jar:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task13qjC6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 18:46:44.722913 15669 mini_ranger_kms.cc:240] Using host: 127.15.77.84
I20260620 18:46:44.726665 15669 mini_ranger_kms.cc:276] Ranger KMS PID: 27889
I20260620 18:46:44.726745 15669 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.77.84:43187
Jun 20, 2026 6:46:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 6:46:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 6:46:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 6:46:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 6:46:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 6:46:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 20 18:46:47 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781981207, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.77.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:47.033738 15669 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.264s user 0.000s sys 0.148s
I20260620 18:46:47.033839 15669 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 18:46:47.033886 15669 mini_ranger_kms.cc:401] 127.15.77.84:43187/kms/v1/keys
Jun 20 18:46:47 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981207, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.77.84@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
Jun 20 18:46:47 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981207, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.77.84@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
Jun 20 18:46:50 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:46:50.186967 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40525
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40525
--ranger_config_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:43187/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:50.278935 27940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:50.279197 27940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:50.279238 27940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:50.280540 27940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:46:50.280615 27940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:50.280642 27940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:50.280673 27940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:46:50.280700 27940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:46:50.282553 27940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.77.84:43187/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40525
--ranger_config_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:50.282833 27940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:50.283068 27940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:50.289242 27946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:50.289242 27948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:50.289610 27940 server_base.cc:1034] running on GCE node
W20260620 18:46:50.289371 27945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:50.289960 27940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:50.290297 27940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:50.291455 27940 hybrid_clock.cc:648] HybridClock initialized: now 1781981210291435 us; error 28 us; skew 500 ppm
Jun 20 18:46:50 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981210, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:50.298274 27940 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:46:50.298919 27940 webserver.cc:460] Webserver started at http://127.15.77.126:45627/ using document root <none> and password file <none>
I20260620 18:46:50.299165 27940 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:50.299225 27940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:50.299396 27940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 18:46:50 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981210, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:50.348042 27940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde"
format_stamp: "Formatted at 2026-06-20 18:46:50 on dist-test-slave-tvl9"
server_key: "07530f721f24672fbbd7bfcfb107c35c"
server_key_iv: "b7f7004512573778e168ac355bb785f3"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:50.348556 27940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde"
format_stamp: "Formatted at 2026-06-20 18:46:50 on dist-test-slave-tvl9"
server_key: "07530f721f24672fbbd7bfcfb107c35c"
server_key_iv: "b7f7004512573778e168ac355bb785f3"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:50.350186 27940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:46:50.363898 27955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:50.364228 27940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 18:46:50.364341 27940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde"
format_stamp: "Formatted at 2026-06-20 18:46:50 on dist-test-slave-tvl9"
server_key: "07530f721f24672fbbd7bfcfb107c35c"
server_key_iv: "b7f7004512573778e168ac355bb785f3"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:50.364473 27940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:50.391307 27940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:50.393424 27940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:50.393599 27940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:50.398664 27940 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:40525
I20260620 18:46:50.398741 28007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:40525 every 8 connection(s)
I20260620 18:46:50.399178 27940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:46:50.400010 28008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:50.402293 28008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: Bootstrap starting.
I20260620 18:46:50.402977 28008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:50.403313 28008 log.cc:824] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:50.403836 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 27940
I20260620 18:46:50.403929 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:46:50.404105 15669 external_mini_cluster.cc:1428] Setting key 2d792558350e4d0591fd95e59b2de976
I20260620 18:46:50.404245 28008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: No bootstrap required, opened a new log
I20260620 18:46:50.407168 28008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } }
I20260620 18:46:50.407447 28008 raft_consensus.cc:374] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:50.407534 28008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210cf47aa91f4c6fbfc6e5a8a4bacbde, State: Initialized, Role: FOLLOWER
I20260620 18:46:50.407683 28008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } }
I20260620 18:46:50.407768 28008 raft_consensus.cc:388] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:46:50.407799 28008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:46:50.407866 28008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:50.408581 28008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } }
I20260620 18:46:50.408692 28008 leader_election.cc:302] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 210cf47aa91f4c6fbfc6e5a8a4bacbde; no voters:
I20260620 18:46:50.408870 28008 leader_election.cc:288] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:46:50.408917 28013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:46:50.409178 28013 raft_consensus.cc:686] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 1 LEADER]: Becoming Leader. State: Replica: 210cf47aa91f4c6fbfc6e5a8a4bacbde, State: Running, Role: LEADER
I20260620 18:46:50.409266 28008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:46:50.409389 28013 consensus_queue.cc:237] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } }
I20260620 18:46:50.409868 28013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } } }
I20260620 18:46:50.410010 28013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:50.410184 28013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 210cf47aa91f4c6fbfc6e5a8a4bacbde. Latest consensus state: current_term: 1 leader_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } } }
I20260620 18:46:50.410295 28013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:50.410478 28022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:46:50.410686 28008 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 18:46:50.411070 28022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 18:46:50 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:46:50.417511 28022 catalog_manager.cc:1347] Generated new cluster ID: 504edbeb6a30447cadc48a1e72c9ae55
I20260620 18:46:50.417745 28022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:46:50.420411 28012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:50.404717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60652 (local address 127.15.77.126:40525)
0620 18:46:50.404904 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:50.404913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:50.404943 (+ 30us) server_negotiation.cc:407] Connection header received
0620 18:46:50.405597 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:50.405604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:50.405655 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:50.405799 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:50.406468 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:50.407471 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:50.408431 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:50.409034 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:50.410988 (+ 1954us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:50.412173 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:50.412178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:50.412178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:50.412192 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:50.417171 (+ 4979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:50.417812 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:50.417815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:50.417816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:50.417994 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:50.418277 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:50.418278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:50.418278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:50.418780 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:50.419557 (+ 777us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:50.419645 (+ 88us) server_negotiation.cc:299] Negotiation successful
0620 18:46:50.420069 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
I20260620 18:46:50.436729 28022 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:46:50.437543 28022 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:46:50.441619 28022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: Generated new TSK 0
I20260620 18:46:50.441802 28022 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:46:51.125507 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:43187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40525
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:51.250429 28053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:51.250738 28053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:51.250784 28053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:51.252592 28053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:51.252661 28053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:51.252704 28053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:46:51.254963 28053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.77.84:43187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:51.255254 28053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:51.255657 28053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:46:51.261164 28053 server_base.cc:1034] running on GCE node
W20260620 18:46:51.261132 28058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:51.261134 28059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:51.261271 28061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:51.261615 28053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:51.261934 28053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:51.263186 28053 hybrid_clock.cc:648] HybridClock initialized: now 1781981211263158 us; error 45 us; skew 500 ppm
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:51.266520 28053 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:46:51.267107 28053 webserver.cc:460] Webserver started at http://127.15.77.65:34041/ using document root <none> and password file <none>
I20260620 18:46:51.267428 28053 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:51.267500 28053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:51.267664 28053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:51.282882 28053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "7ef69a16900441a69288255e9dd34e78"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "993f780c62b7f9ba4b4ed3e2d0109b4c"
server_key_iv: "420e668daa3aafd3e16a2a11e6af27bd"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:51.283530 28053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "7ef69a16900441a69288255e9dd34e78"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "993f780c62b7f9ba4b4ed3e2d0109b4c"
server_key_iv: "420e668daa3aafd3e16a2a11e6af27bd"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:51.285498 28053 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260620 18:46:51.298954 28068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:51.299175 28053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:46:51.299273 28053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "7ef69a16900441a69288255e9dd34e78"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "993f780c62b7f9ba4b4ed3e2d0109b4c"
server_key_iv: "420e668daa3aafd3e16a2a11e6af27bd"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:51.299391 28053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:51.321086 28053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:51.323699 28053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:51.323859 28053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:51.324163 28053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:46:51.324589 28053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:46:51.324666 28053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:51.324724 28053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:46:51.324769 28053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:51.332311 28053 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:41137
I20260620 18:46:51.332844 28053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:46:51.333577 28183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:41137 every 8 connection(s)
I20260620 18:46:51.334242 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 28053
I20260620 18:46:51.334337 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:46:51.334564 15669 external_mini_cluster.cc:1428] Setting key b3155226489dd3906164f9c8fa3ab166
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:46:51.345836 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:51.334266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40525 (local address 127.15.77.65:58413)
0620 18:46:51.334447 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:51.334491 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:51.335431 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:51.335856 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:51.335861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:51.336104 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:51.336381 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:51.336387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.337420 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.337421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:51.338701 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:51.338704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.339257 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.339258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:51.339467 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:51.340106 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:51.340118 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:51.341807 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:51.344106 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:51.344115 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:51.344118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:51.344504 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:51.344772 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:51.344773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:51.344773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:51.344943 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:51.345285 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:51.345291 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:51.345426 (+ 135us) client_negotiation.cc:769] Sending connection context
0620 18:46:51.345520 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 18:46:51.345650 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":109,"threads_started":1}
I20260620 18:46:51.345902 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:51.334264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:58413 (local address 127.15.77.126:40525)
0620 18:46:51.335608 (+ 1344us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:51.335610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:51.335617 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:46:51.335638 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:51.335638 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:51.335662 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:51.335724 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:51.336490 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.337303 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.338813 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.339095 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.339143 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:51.341940 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:51.341942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:51.341942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:51.341950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:51.343988 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:51.344601 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:51.344603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:51.344604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:51.344680 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:51.345029 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:51.345030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:51.345031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:51.345198 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:51.345281 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:51.345605 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 18:46:51.345775 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1312,"thread_start_us":91,"threads_started":1}
I20260620 18:46:51.346518 28184 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40525
I20260620 18:46:51.346634 28184 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:51.346879 28184 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
I20260620 18:46:51.347661 27972 ts_manager.cc:194] Registered new tserver with Master: 7ef69a16900441a69288255e9dd34e78 (127.15.77.65:41137)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
I20260620 18:46:51.348654 27972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:58413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:46:51.364228 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:43187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40525
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:51.479060 28190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:51.479374 28190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:51.479439 28190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:51.481230 28190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:51.481293 28190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:51.481336 28190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:46:51.483596 28190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.77.84:43187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:51.483884 28190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:51.484220 28190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:51.490829 28197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:51.490837 28200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:51.491204 28198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:51.491497 28190 server_base.cc:1034] running on GCE node
I20260620 18:46:51.491736 28190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:51.492031 28190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:51.493216 28190 hybrid_clock.cc:648] HybridClock initialized: now 1781981211493190 us; error 39 us; skew 500 ppm
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:51.496778 28190 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:46:51.497347 28190 webserver.cc:460] Webserver started at http://127.15.77.66:38999/ using document root <none> and password file <none>
I20260620 18:46:51.497637 28190 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:51.497714 28190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:51.497900 28190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:51.511444 28190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "dbcf11af0a6241389eef679e4043af1f"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "6ed7bd3b3fc2c5981e836d471f6d9fe1"
server_key_iv: "ee5d40715a8de9922f529647e20b3f6b"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:51.511849 28190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "dbcf11af0a6241389eef679e4043af1f"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "6ed7bd3b3fc2c5981e836d471f6d9fe1"
server_key_iv: "ee5d40715a8de9922f529647e20b3f6b"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:51.513305 28190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:46:51.526875 28207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:51.527107 28190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260620 18:46:51.527215 28190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "dbcf11af0a6241389eef679e4043af1f"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "6ed7bd3b3fc2c5981e836d471f6d9fe1"
server_key_iv: "ee5d40715a8de9922f529647e20b3f6b"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:51.527392 28190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:51.551760 28190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:51.555333 28190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:51.555559 28190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:51.555845 28190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:46:51.556325 28190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:46:51.558455 28190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 18:46:51.558509 28190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:46:51.558537 28190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:51.565213 28190 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:44391
I20260620 18:46:51.565665 28190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:46:51.571872 28320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:44391 every 8 connection(s)
I20260620 18:46:51.571942 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 28190
I20260620 18:46:51.572023 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:46:51.572239 15669 external_mini_cluster.cc:1428] Setting key 44fd971115e8efb234a9476d3547b5cb
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
I20260620 18:46:51.587431 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:51.573480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:35385 (local address 127.15.77.126:40525)
0620 18:46:51.573523 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:51.573525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:51.575619 (+ 2094us) server_negotiation.cc:407] Connection header received
0620 18:46:51.576304 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:51.576306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:51.576321 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:51.576352 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:51.577182 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.577971 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.579498 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.579807 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.579850 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:51.582601 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:51.582603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:51.582603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:51.582611 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:51.585040 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:51.585561 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:51.585562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:51.585563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:51.585626 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:51.586547 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:51.586548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:51.586549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:51.586707 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:51.586762 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:51.587195 (+ 433us) server_negotiation.cc:299] Negotiation successful
0620 18:46:51.587297 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:46:51.588125 28323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:51.572854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40525 (local address 127.15.77.66:35385)
0620 18:46:51.575469 (+ 2615us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:51.575523 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:51.576189 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:51.576538 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:51.576542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:51.576797 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:51.577068 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:51.577072 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.578080 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.578080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:51.579371 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:51.579374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.579959 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.579960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:51.580106 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:51.580788 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:51.580799 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:51.582481 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:51.585140 (+ 2659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:51.585149 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:51.585152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:51.585478 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:51.586039 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:51.586039 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:51.586040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:51.586211 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:51.586816 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:51.586823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:51.587011 (+ 188us) client_negotiation.cc:769] Sending connection context
0620 18:46:51.587674 (+ 663us) client_negotiation.cc:240] Negotiation successful
0620 18:46:51.587863 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2561,"thread_start_us":106,"threads_started":1}
I20260620 18:46:51.588631 28321 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40525
I20260620 18:46:51.588755 28321 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:51.588969 28321 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
I20260620 18:46:51.589359 27972 ts_manager.cc:194] Registered new tserver with Master: dbcf11af0a6241389eef679e4043af1f (127.15.77.66:44391)
I20260620 18:46:51.589864 27972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:35385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:46:51.606684 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:43187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40525
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:51.733125 28327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:51.733505 28327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:51.733596 28327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:51.735675 28327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:51.735810 28327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:51.735898 28327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:46:51.738318 28327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.77.84:43187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:51.738953 28327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:51.739300 28327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:51.746587 28332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:51.747305 28335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:51.747903 28327 server_base.cc:1034] running on GCE node
W20260620 18:46:51.748116 28333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:51.748420 28327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:51.748754 28327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:51.749939 28327 hybrid_clock.cc:648] HybridClock initialized: now 1781981211749912 us; error 38 us; skew 500 ppm
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:51.753294 28327 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:46:51.753834 28327 webserver.cc:460] Webserver started at http://127.15.77.67:46295/ using document root <none> and password file <none>
I20260620 18:46:51.754086 28327 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:51.754132 28327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:51.754261 28327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:51.775688 28327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "09bb0b6abab042628738c05c0a8c1155"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "7ff80718956685b741afdd2a713b1893"
server_key_iv: "4debefc4daf3c7d74593636725771044"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:51.782843 28327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "09bb0b6abab042628738c05c0a8c1155"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "7ff80718956685b741afdd2a713b1893"
server_key_iv: "4debefc4daf3c7d74593636725771044"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:51.787875 28327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.002s
I20260620 18:46:51.802812 28342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:51.803103 28327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 18:46:51.803212 28327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "09bb0b6abab042628738c05c0a8c1155"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "7ff80718956685b741afdd2a713b1893"
server_key_iv: "4debefc4daf3c7d74593636725771044"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:51.803344 28327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:51.817031 28327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:51.819402 28327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:51.819588 28327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:51.819862 28327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:46:51.820257 28327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:46:51.820312 28327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:51.820384 28327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:46:51.820413 28327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:51.827991 28327 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:38605
I20260620 18:46:51.828377 28327 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:46:51.828627 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 28327
I20260620 18:46:51.828722 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:46:51.828934 15669 external_mini_cluster.cc:1428] Setting key 55d22d32bf4caf9d6b85f7005b1132b9
I20260620 18:46:51.829841 28455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:38605 every 8 connection(s)
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:46:51.838876 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:51.829742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:51275 (local address 127.15.77.126:40525)
0620 18:46:51.829796 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:51.829799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:51.830160 (+ 361us) server_negotiation.cc:407] Connection header received
0620 18:46:51.830726 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:51.830727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:51.830740 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:51.830794 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:51.831555 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.832132 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.833247 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.833460 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.833549 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:51.835855 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:51.835858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:51.835858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:51.835867 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:51.837345 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:51.837844 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:51.837845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:51.837846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:51.837915 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:51.838187 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:51.838188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:51.838189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:51.838314 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:51.838395 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:51.838694 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 18:46:51.838780 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:46:51.839097 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:51.829494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40525 (local address 127.15.77.67:51275)
0620 18:46:51.830037 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:51.830069 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:51.830590 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:51.830917 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:51.830921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:51.831175 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:51.831462 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:51.831467 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.832300 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.832301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:51.833136 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:51.833138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:51.833574 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:51.833575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:51.833707 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:51.834316 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:51.834329 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:51.835751 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:51.837455 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:51.837464 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:51.837467 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:51.837760 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:51.837986 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:51.837987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:51.837987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:51.838114 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:51.838386 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:51.838394 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:51.838552 (+ 158us) client_negotiation.cc:769] Sending connection context
0620 18:46:51.838670 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 18:46:51.838832 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":86,"threads_started":1}
I20260620 18:46:51.839771 28456 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40525
I20260620 18:46:51.839900 28456 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:51.840133 28456 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
I20260620 18:46:51.840595 27972 ts_manager.cc:194] Registered new tserver with Master: 09bb0b6abab042628738c05c0a8c1155 (127.15.77.67:38605)
I20260620 18:46:51.840974 27972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:51275
I20260620 18:46:51.851490 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:46:51.852490 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 28053
I20260620 18:46:51.860716 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 28190
I20260620 18:46:51.866240 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 28327
I20260620 18:46:51.872864 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 27940
I20260620 18:46:51.880234 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40525
--webserver_interface=127.15.77.126
--webserver_port=45627
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40525
--ranger_config_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:43187/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.77.84:43187/kms with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:51.963517 28459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:51.963764 28459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:51.963802 28459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:51.964994 28459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:46:51.965062 28459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:51.965090 28459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:51.965119 28459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:46:51.965142 28459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:46:51.966758 28459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.77.84:43187/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40525
--ranger_config_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=45627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:51.967041 28459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:51.967268 28459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:51.972234 28467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:51.972277 28464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:51.972528 28459 server_base.cc:1034] running on GCE node
W20260620 18:46:51.972235 28465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:51.972815 28459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:51.973127 28459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:51.974299 28459 hybrid_clock.cc:648] HybridClock initialized: now 1781981211974275 us; error 40 us; skew 500 ppm
Jun 20 18:46:51 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:51.976876 28459 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:46:51.977391 28459 webserver.cc:460] Webserver started at http://127.15.77.126:45627/ using document root <none> and password file <none>
I20260620 18:46:51.977632 28459 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:51.977692 28459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 18:46:52 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981211, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:52.055626 28459 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:46:52.056519 28475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:52.056747 28459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:46:52.056864 28459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde"
format_stamp: "Formatted at 2026-06-20 18:46:50 on dist-test-slave-tvl9"
server_key: "07530f721f24672fbbd7bfcfb107c35c"
server_key_iv: "b7f7004512573778e168ac355bb785f3"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:52.057335 28459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:52.092026 28459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:52.094127 28459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:52.094282 28459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:52.098793 28459 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:40525
I20260620 18:46:52.098845 28527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:40525 every 8 connection(s)
I20260620 18:46:52.099220 28459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:46:52.100864 28528 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:46:52.101725 28528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: Bootstrap starting.
I20260620 18:46:52.107782 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 28459
I20260620 18:46:52.108091 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:43187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:41137
--local_ip_for_outbound_sockets=127.15.77.65
--tserver_master_addrs=127.15.77.126:40525
--webserver_port=34041
--webserver_interface=127.15.77.65
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.77.84:43187/kms with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:46:52.111905 28528 log.cc:824] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:52.114404 28528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:46:52.114717 28528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: Bootstrap complete.
I20260620 18:46:52.118130 28528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } }
I20260620 18:46:52.118463 28528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210cf47aa91f4c6fbfc6e5a8a4bacbde, State: Initialized, Role: FOLLOWER
I20260620 18:46:52.118682 28528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } }
I20260620 18:46:52.118767 28528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:46:52.118842 28528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:46:52.118902 28528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 1 FOLLOWER]: Advancing to term 2
I20260620 18:46:52.120347 28528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } }
I20260620 18:46:52.120461 28528 leader_election.cc:302] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 210cf47aa91f4c6fbfc6e5a8a4bacbde; no voters:
I20260620 18:46:52.120693 28528 leader_election.cc:288] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 18:46:52.120761 28532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 2 FOLLOWER]: Leader election won for term 2
I20260620 18:46:52.120960 28532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [term 2 LEADER]: Becoming Leader. State: Replica: 210cf47aa91f4c6fbfc6e5a8a4bacbde, State: Running, Role: LEADER
I20260620 18:46:52.121026 28528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:46:52.121157 28532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } }
I20260620 18:46:52.121495 28532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } } }
I20260620 18:46:52.121616 28532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:52.121793 28532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 210cf47aa91f4c6fbfc6e5a8a4bacbde. Latest consensus state: current_term: 2 leader_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210cf47aa91f4c6fbfc6e5a8a4bacbde" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40525 } } }
I20260620 18:46:52.121886 28532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:52.122097 28540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:46:52.122625 28540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:46:52.122778 28540 catalog_manager.cc:1259] Loaded cluster ID: 504edbeb6a30447cadc48a1e72c9ae55
I20260620 18:46:52.122855 28540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:46:52.123106 28528 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 18:46:52.124156 28540 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:46:52.124337 28540 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: Loaded TSK: 0
I20260620 18:46:52.124548 28540 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 18:46:52.226042 28530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:52.226434 28530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:52.226524 28530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:52.228329 28530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:52.228454 28530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:52.228516 28530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:46:52.230670 28530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.77.84:43187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:41137
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=34041
--tserver_master_addrs=127.15.77.126:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:52.230921 28530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:52.231254 28530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:52.238080 28566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:52.238281 28565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:52.238171 28568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:52.238513 28530 server_base.cc:1034] running on GCE node
I20260620 18:46:52.238726 28530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:52.239035 28530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:52.240221 28530 hybrid_clock.cc:648] HybridClock initialized: now 1781981212240189 us; error 55 us; skew 500 ppm
Jun 20 18:46:52 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981212, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:52.243626 28530 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:46:52.244200 28530 webserver.cc:460] Webserver started at http://127.15.77.65:34041/ using document root <none> and password file <none>
I20260620 18:46:52.244510 28530 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:52.244596 28530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 18:46:52 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981212, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:52.270542 28530 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260620 18:46:52.271775 28576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:52.272024 28530 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:46:52.272121 28530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "7ef69a16900441a69288255e9dd34e78"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "993f780c62b7f9ba4b4ed3e2d0109b4c"
server_key_iv: "420e668daa3aafd3e16a2a11e6af27bd"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:52.272617 28530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:52.298103 28530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:52.300774 28530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:52.300961 28530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:52.301288 28530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:46:52.301684 28530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:46:52.301757 28530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:52.301808 28530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:46:52.301841 28530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:52.308512 28530 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:41137
I20260620 18:46:52.309089 28530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:46:52.315050 28689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:41137 every 8 connection(s)
I20260620 18:46:52.318315 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 28530
I20260620 18:46:52.318629 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:43187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:44391
--local_ip_for_outbound_sockets=127.15.77.66
--tserver_master_addrs=127.15.77.126:40525
--webserver_port=38999
--webserver_interface=127.15.77.66
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.77.84:43187/kms with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 18:46:52 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981212, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:46:52.329456 28692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:52.315821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40525 (local address 127.15.77.65:58987)
0620 18:46:52.316329 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:52.316382 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:52.317038 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:52.317607 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:52.317612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:52.317862 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:52.318170 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:52.318177 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.319968 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.319969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:52.321277 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:52.321280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.322206 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.322208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:52.322350 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:52.322994 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:52.323006 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:52.324605 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:52.327168 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:52.327177 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:52.327180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:52.327569 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:52.327903 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:52.327904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:52.327905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:52.328055 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:52.328624 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:52.328630 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:52.328823 (+ 193us) client_negotiation.cc:769] Sending connection context
0620 18:46:52.328945 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 18:46:52.329136 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":100,"threads_started":1}
I20260620 18:46:52.330678 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:52.316227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:58987 (local address 127.15.77.126:40525)
0620 18:46:52.316447 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:52.316457 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:52.316496 (+ 39us) server_negotiation.cc:407] Connection header received
0620 18:46:52.317323 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:52.317331 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:52.317397 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:52.317551 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:52.318318 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.319826 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.321416 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.322078 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.322197 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:52.324754 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:52.324760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:52.324760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:52.324773 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:52.327055 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:52.327689 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:52.327693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:52.327693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:52.327819 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:52.328135 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:52.328136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:52.328136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:52.328496 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:52.328590 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:52.330235 (+ 1645us) server_negotiation.cc:299] Negotiation successful
0620 18:46:52.330397 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":87,"threads_started":1}
I20260620 18:46:52.331789 28690 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40525
I20260620 18:46:52.331919 28690 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:52.332145 28690 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
W20260620 18:46:52.333069 28690 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:40525 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 18:46:52.333138 28690 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:52.333196 28690 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
I20260620 18:46:52.333612 28690 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:52.333683 28690 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
W20260620 18:46:52.334010 28690 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 18:46:52.439103 28694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:52.439456 28694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:52.439497 28694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:52.441138 28694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:52.441193 28694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:52.441226 28694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:46:52.442920 28694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.77.84:43187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:44391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=38999
--tserver_master_addrs=127.15.77.126:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:52.443221 28694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:52.443543 28694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:52.448673 28699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:52.448688 28700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:52.448881 28702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:52.450009 28694 server_base.cc:1034] running on GCE node
I20260620 18:46:52.450268 28694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:52.450558 28694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:52.451769 28694 hybrid_clock.cc:648] HybridClock initialized: now 1781981212451743 us; error 42 us; skew 500 ppm
Jun 20 18:46:52 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981212, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:52.454530 28694 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:46:52.455068 28694 webserver.cc:460] Webserver started at http://127.15.77.66:38999/ using document root <none> and password file <none>
I20260620 18:46:52.455359 28694 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:52.455441 28694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 18:46:52 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981212, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:52.473476 28694 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260620 18:46:52.475831 28710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:52.476020 28694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260620 18:46:52.476132 28694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "dbcf11af0a6241389eef679e4043af1f"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "6ed7bd3b3fc2c5981e836d471f6d9fe1"
server_key_iv: "ee5d40715a8de9922f529647e20b3f6b"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:52.476688 28694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:52.516623 28694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:52.518815 28694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:52.518993 28694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:52.519268 28694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:46:52.519718 28694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:46:52.519758 28694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:52.519809 28694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:46:52.519837 28694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:52.527467 28694 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:44391
I20260620 18:46:52.527567 28823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:44391 every 8 connection(s)
I20260620 18:46:52.527956 28694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:46:52.529621 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 28694
I20260620 18:46:52.529904 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.77.84:43187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:38605
--local_ip_for_outbound_sockets=127.15.77.67
--tserver_master_addrs=127.15.77.126:40525
--webserver_port=46295
--webserver_interface=127.15.77.67
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.77.84:43187/kms with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 18:46:52 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981212, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:46:52.542394 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:52.528836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:38191 (local address 127.15.77.126:40525)
0620 18:46:52.528913 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:52.528917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:52.529120 (+ 203us) server_negotiation.cc:407] Connection header received
0620 18:46:52.531668 (+ 2548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:52.531670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:52.531685 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:52.531719 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:52.532565 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.533373 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.535052 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.535458 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.535511 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:52.538447 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:52.538449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:52.538450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:52.538459 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:52.540450 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:52.541051 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:52.541053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:52.541053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:52.541121 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:52.541456 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:52.541457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:52.541457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:52.541623 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:52.541676 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:52.542143 (+ 467us) server_negotiation.cc:299] Negotiation successful
0620 18:46:52.542248 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 18:46:52.543365 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:52.528828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40525 (local address 127.15.77.66:38191)
0620 18:46:52.528993 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:52.529038 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:52.531507 (+ 2469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:52.531925 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:52.531930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:52.532192 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:52.532451 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:52.532456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.533488 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.533490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:52.534921 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:52.534924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.535635 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.535636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:52.535800 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:52.536436 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:52.536450 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:52.538320 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:52.540561 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:52.540572 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:52.540575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:52.540936 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:52.541205 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:52.541206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:52.541206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:52.541367 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:52.541732 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:52.541739 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:52.541939 (+ 200us) client_negotiation.cc:769] Sending connection context
0620 18:46:52.542810 (+ 871us) client_negotiation.cc:240] Negotiation successful
0620 18:46:52.543013 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260620 18:46:52.544055 28824 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40525
I20260620 18:46:52.544175 28824 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:52.544400 28824 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
W20260620 18:46:52.545192 28824 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:40525 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 18:46:52.545281 28824 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:52.545396 28824 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
I20260620 18:46:52.545791 28824 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:52.545905 28824 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
W20260620 18:46:52.546242 28824 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 18:46:52.638492 28827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:52.638890 28827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:52.638981 28827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:52.640838 28827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:52.640923 28827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:52.640973 28827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:46:52.642575 28827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.77.84:43187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:38605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=46295
--tserver_master_addrs=127.15.77.126:40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:52.642861 28827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:52.643103 28827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:52.648815 28833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:52.648815 28834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:52.649134 28836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:52.649372 28827 server_base.cc:1034] running on GCE node
I20260620 18:46:52.649575 28827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:52.649802 28827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:52.650947 28827 hybrid_clock.cc:648] HybridClock initialized: now 1781981212650907 us; error 57 us; skew 500 ppm
Jun 20 18:46:52 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981212, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:52.653682 28827 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:46:52.654207 28827 webserver.cc:460] Webserver started at http://127.15.77.67:46295/ using document root <none> and password file <none>
I20260620 18:46:52.654481 28827 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:52.654577 28827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 18:46:52 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981212, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:52.669709 28827 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:46:52.670785 28844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:52.671012 28827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 18:46:52.671113 28827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "09bb0b6abab042628738c05c0a8c1155"
format_stamp: "Formatted at 2026-06-20 18:46:51 on dist-test-slave-tvl9"
server_key: "7ff80718956685b741afdd2a713b1893"
server_key_iv: "4debefc4daf3c7d74593636725771044"
server_key_version: "kuduclusterkey@0"
I20260620 18:46:52.672040 28827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:52.684350 28827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:52.686607 28827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:52.686818 28827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:52.687124 28827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:46:52.687577 28827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:46:52.687628 28827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:52.687685 28827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:46:52.687726 28827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:52.693492 28827 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:38605
I20260620 18:46:52.693921 28827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:46:52.696671 28957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:38605 every 8 connection(s)
I20260620 18:46:52.700043 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 28827
Jun 20 18:46:52 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981212, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:46:52.708828 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:52.700471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37470 (local address 127.15.77.126:40525)
0620 18:46:52.700644 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:52.700647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:52.700676 (+ 29us) server_negotiation.cc:407] Connection header received
0620 18:46:52.700758 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:52.700759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:52.700770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:52.700858 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:52.701445 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.701929 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.703106 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.703469 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.703555 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:52.705173 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:52.705175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:52.705175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:52.705182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:52.707217 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:52.707968 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:52.707968 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:52.707969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:52.708024 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:52.708338 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:52.708338 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:52.708339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:52.708413 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:52.708585 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:52.708625 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 18:46:52.708712 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":53,"threads_started":1}
I20260620 18:46:52.709465 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:52.697687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:48909 (local address 127.15.77.126:40525)
0620 18:46:52.697739 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:52.697742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:52.698107 (+ 365us) server_negotiation.cc:407] Connection header received
0620 18:46:52.698848 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:52.698850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:52.698863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:52.698894 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:52.699774 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.700544 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.702100 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.702394 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.702445 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:52.705324 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:52.705325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:52.705325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:52.705331 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:52.707067 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:52.707630 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:52.707631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:52.707631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:52.707696 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:52.708141 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:52.708141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:52.708142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:52.708271 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:52.708321 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:52.709307 (+ 986us) server_negotiation.cc:299] Negotiation successful
0620 18:46:52.709382 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:46:52.709688 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:52.697399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40525 (local address 127.15.77.67:48909)
0620 18:46:52.697960 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:52.698009 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:52.698712 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:52.699104 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:52.699109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:52.699388 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:52.699660 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:52.699665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.700679 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.700680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:52.701960 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:52.701963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:52.702570 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:52.702571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:52.702715 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:52.703387 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:52.703399 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:52.705229 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:52.707163 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:52.707170 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:52.707173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:52.707529 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:52.707769 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:52.707769 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:52.707770 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:52.707909 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:52.708966 (+ 1057us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:52.708974 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:52.709160 (+ 186us) client_negotiation.cc:769] Sending connection context
0620 18:46:52.709270 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 18:46:52.709428 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":83,"threads_started":1}
I20260620 18:46:52.710155 28958 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40525
I20260620 18:46:52.710279 28958 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:52.710467 28958 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
W20260620 18:46:52.711108 28958 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:40525 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 18:46:52.711195 28958 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:52.711269 28958 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
I20260620 18:46:52.711586 28958 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:52.711658 28958 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
W20260620 18:46:52.711984 28958 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 18:46:53 dist-test-slave-tvl9 krb5kdc[27496](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781981213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:53.334205 28690 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:53.334398 28690 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
I20260620 18:46:53.334846 28492 ts_manager.cc:194] Registered new tserver with Master: 7ef69a16900441a69288255e9dd34e78 (127.15.77.65:41137)
I20260620 18:46:53.335677 28492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:58987
I20260620 18:46:53.546474 28824 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:53.546628 28824 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
I20260620 18:46:53.546998 28492 ts_manager.cc:194] Registered new tserver with Master: dbcf11af0a6241389eef679e4043af1f (127.15.77.66:44391)
I20260620 18:46:53.547569 28492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:38191
Jun 20 18:46:53 dist-test-slave-tvl9 krb5kdc[27496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981213, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for HTTP/127.15.77.84@KRBTEST.COM
I20260620 18:46:53.712200 28958 heartbeater.cc:464] Registering TS with master...
I20260620 18:46:53.712379 28958 heartbeater.cc:511] Master 127.15.77.126:40525 requested a full tablet report, sending...
I20260620 18:46:53.712756 28492 ts_manager.cc:194] Registered new tserver with Master: 09bb0b6abab042628738c05c0a8c1155 (127.15.77.67:38605)
I20260620 18:46:53.713294 28492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:48909
I20260620 18:46:53.718173 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:46:53.728916 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.719161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37476 (local address 127.15.77.126:40525)
0620 18:46:53.719284 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:53.719287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:53.721202 (+ 1915us) server_negotiation.cc:407] Connection header received
0620 18:46:53.721265 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:53.721266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:53.721279 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:53.721351 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:53.722105 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.722903 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.723997 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.724311 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.724392 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.725922 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:53.725923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:53.725924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:53.725932 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:53.727721 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:53.728115 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:53.728117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:53.728117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:53.728183 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:53.728399 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:53.728400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:53.728400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:53.728530 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:53.728635 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:53.728738 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 18:46:53.728818 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260620 18:46:53.731567 28492 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:46:53.733857 28492 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:46:53.743938 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.737061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:44391 (local address 127.0.0.1:49104)
0620 18:46:53.739426 (+ 2365us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:53.739438 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:53.739809 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:53.740089 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:53.740090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:53.740093 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:46:53.740130 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:53.740131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.741356 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.741357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:53.742556 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:53.742558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.743616 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.743617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:53.743702 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.743718 (+ 16us) client_negotiation.cc:769] Sending connection context
0620 18:46:53.743790 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 18:46:53.743842 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2334,"thread_start_us":109,"threads_started":1}
I20260620 18:46:53.744307 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.737665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.15.77.67:38605)
0620 18:46:53.737887 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:53.737892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:53.739970 (+ 2078us) server_negotiation.cc:407] Connection header received
0620 18:46:53.740083 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:53.740085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:53.740102 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:53.740157 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:46:53.740407 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.741363 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.743049 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.743857 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.743904 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.743957 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:53.744188 (+ 231us) server_negotiation.cc:299] Negotiation successful
0620 18:46:53.744230 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":56,"threads_started":1}
I20260620 18:46:53.744517 28985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.737299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:38605 (local address 127.0.0.1:55414)
0620 18:46:53.739908 (+ 2609us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:53.739918 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:53.739998 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:53.740268 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:53.740269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:53.740271 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:46:53.740325 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:53.740327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.741470 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.741471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:53.742947 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:53.742950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.744018 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.744019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:53.744094 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.744098 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:46:53.744403 (+ 305us) client_negotiation.cc:240] Negotiation successful
0620 18:46:53.744452 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2593,"thread_start_us":68,"threads_started":1}
I20260620 18:46:53.744738 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.737219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49104 (local address 127.15.77.66:44391)
0620 18:46:53.737361 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:53.737367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:53.739517 (+ 2150us) server_negotiation.cc:407] Connection header received
0620 18:46:53.739902 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:53.739904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:53.739922 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:53.739997 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:46:53.740216 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.741260 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.742655 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.743453 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.743520 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.743580 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:53.744527 (+ 947us) server_negotiation.cc:299] Negotiation successful
0620 18:46:53.744572 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260620 18:46:53.745425 28758 tablet_service.cc:1467] Processing CreateTablet for tablet 117639d4110a4b63b058bb75ca50a5ff (DEFAULT_TABLE table=test-table [id=13461b67cab54303b09efe8bd3f1b922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:46:53.745818 28758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 117639d4110a4b63b058bb75ca50a5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:53.748041 28892 tablet_service.cc:1467] Processing CreateTablet for tablet 117639d4110a4b63b058bb75ca50a5ff (DEFAULT_TABLE table=test-table [id=13461b67cab54303b09efe8bd3f1b922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:46:53.748433 28892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 117639d4110a4b63b058bb75ca50a5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:53.749713 28991 tablet_bootstrap.cc:492] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: Bootstrap starting.
I20260620 18:46:53.750296 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.737437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.15.77.65:41137)
0620 18:46:53.737706 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:53.737711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:53.737721 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:46:53.737757 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:53.737759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:53.737776 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:53.737851 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:46:53.738113 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.739101 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.748962 (+ 9861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.749769 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.749825 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.749887 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:53.750165 (+ 278us) server_negotiation.cc:299] Negotiation successful
0620 18:46:53.750212 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":66,"threads_started":1}
I20260620 18:46:53.750365 28991 tablet_bootstrap.cc:654] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:53.750655 28991 log.cc:824] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:53.750674 28988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.737058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:41137 (local address 127.0.0.1:37714)
0620 18:46:53.737421 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:53.737465 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:53.737534 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:53.737946 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:53.737949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:53.737951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:46:53.738021 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:53.738023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.739197 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.739198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:53.748844 (+ 9646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:53.748847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.750001 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.750002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:53.750066 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.750069 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:46:53.750532 (+ 463us) client_negotiation.cc:240] Negotiation successful
0620 18:46:53.750603 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":97,"threads_started":1}
I20260620 18:46:53.751631 28991 tablet_bootstrap.cc:492] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: No bootstrap required, opened a new log
I20260620 18:46:53.751703 28991 ts_tablet_manager.cc:1397] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:46:53.751953 28610 tablet_service.cc:1467] Processing CreateTablet for tablet 117639d4110a4b63b058bb75ca50a5ff (DEFAULT_TABLE table=test-table [id=13461b67cab54303b09efe8bd3f1b922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:46:53.752146 28993 tablet_bootstrap.cc:492] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: Bootstrap starting.
I20260620 18:46:53.752277 28610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 117639d4110a4b63b058bb75ca50a5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:53.752802 28993 tablet_bootstrap.cc:654] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:53.753077 28993 log.cc:824] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:53.753805 28993 tablet_bootstrap.cc:492] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: No bootstrap required, opened a new log
I20260620 18:46:53.753862 28993 ts_tablet_manager.cc:1397] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:46:53.754535 28991 raft_consensus.cc:348] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 } } peers { permanent_uuid: "dbcf11af0a6241389eef679e4043af1f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44391 } } peers { permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 } }
I20260620 18:46:53.754724 28991 raft_consensus.cc:374] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:53.754755 28991 raft_consensus.cc:729] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbcf11af0a6241389eef679e4043af1f, State: Initialized, Role: FOLLOWER
I20260620 18:46:53.755476 28991 consensus_queue.cc:260] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 } } peers { permanent_uuid: "dbcf11af0a6241389eef679e4043af1f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44391 } } peers { permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 } }
I20260620 18:46:53.755880 28991 ts_tablet_manager.cc:1428] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260620 18:46:53.756060 28824 heartbeater.cc:503] Master 127.15.77.126:40525 was elected leader, sending a full tablet report...
I20260620 18:46:53.756501 28993 raft_consensus.cc:348] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 } } peers { permanent_uuid: "dbcf11af0a6241389eef679e4043af1f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44391 } } peers { permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 } }
I20260620 18:46:53.756656 28993 raft_consensus.cc:374] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:53.756690 28993 raft_consensus.cc:729] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bb0b6abab042628738c05c0a8c1155, State: Initialized, Role: FOLLOWER
I20260620 18:46:53.756819 28993 consensus_queue.cc:260] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 } } peers { permanent_uuid: "dbcf11af0a6241389eef679e4043af1f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44391 } } peers { permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 } }
I20260620 18:46:53.757079 28993 ts_tablet_manager.cc:1428] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:46:53.757243 28958 heartbeater.cc:503] Master 127.15.77.126:40525 was elected leader, sending a full tablet report...
I20260620 18:46:53.759477 28995 tablet_bootstrap.cc:492] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78: Bootstrap starting.
I20260620 18:46:53.760143 28995 tablet_bootstrap.cc:654] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:53.760460 28995 log.cc:824] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:53.761358 28995 tablet_bootstrap.cc:492] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78: No bootstrap required, opened a new log
I20260620 18:46:53.761447 28995 ts_tablet_manager.cc:1397] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:46:53.763662 28995 raft_consensus.cc:348] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 } } peers { permanent_uuid: "dbcf11af0a6241389eef679e4043af1f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44391 } } peers { permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 } }
I20260620 18:46:53.763819 28995 raft_consensus.cc:374] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:53.763862 28995 raft_consensus.cc:729] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef69a16900441a69288255e9dd34e78, State: Initialized, Role: FOLLOWER
I20260620 18:46:53.763993 28995 consensus_queue.cc:260] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 } } peers { permanent_uuid: "dbcf11af0a6241389eef679e4043af1f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44391 } } peers { permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 } }
I20260620 18:46:53.764283 28995 ts_tablet_manager.cc:1428] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:46:53.764318 28690 heartbeater.cc:503] Master 127.15.77.126:40525 was elected leader, sending a full tablet report...
W20260620 18:46:53.779196 28825 tablet.cc:2369] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:46:53.779399 28996 raft_consensus.cc:482] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:46:53.779517 28996 raft_consensus.cc:504] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 } } peers { permanent_uuid: "dbcf11af0a6241389eef679e4043af1f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44391 } } peers { permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 } }
I20260620 18:46:53.779919 28996 leader_election.cc:288] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ef69a16900441a69288255e9dd34e78 (127.15.77.65:41137), 09bb0b6abab042628738c05c0a8c1155 (127.15.77.67:38605)
I20260620 18:46:53.784335 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.780339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:35603 (local address 127.15.77.67:38605)
0620 18:46:53.780381 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:53.780383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:53.780797 (+ 414us) server_negotiation.cc:407] Connection header received
0620 18:46:53.780894 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:53.780896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:53.780906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:53.780935 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:46:53.781166 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.781877 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.783307 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.783883 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.783933 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.783941 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:53.784202 (+ 261us) server_negotiation.cc:299] Negotiation successful
0620 18:46:53.784247 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:46:53.784503 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.780310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:59139 (local address 127.15.77.65:41137)
0620 18:46:53.780355 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:53.780357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:53.780654 (+ 297us) server_negotiation.cc:407] Connection header received
0620 18:46:53.780713 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:53.780714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:53.780724 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:53.780782 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:46:53.780993 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.781776 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.783288 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.783877 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.783923 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.783932 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:53.784319 (+ 387us) server_negotiation.cc:299] Negotiation successful
0620 18:46:53.784374 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:46:53.785130 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.780134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:38605 (local address 127.15.77.66:35603)
0620 18:46:53.780745 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:53.780754 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:53.780819 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:53.781000 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:53.781001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:53.781003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:46:53.781074 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:53.781075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.782011 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.782012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:53.783203 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:53.783207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.784021 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.784022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:53.784111 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.784115 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:46:53.784979 (+ 864us) client_negotiation.cc:240] Negotiation successful
0620 18:46:53.785029 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":86,"threads_started":1}
I20260620 18:46:53.785130 29001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.780082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:41137 (local address 127.15.77.66:59139)
0620 18:46:53.780571 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:53.780583 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:53.780656 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:53.780838 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:53.780840 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:53.780842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:46:53.780921 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:53.780923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.781895 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.781895 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:53.783184 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:53.783187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.784009 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.784010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:53.784078 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.784082 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:46:53.784937 (+ 855us) client_negotiation.cc:240] Negotiation successful
0620 18:46:53.784991 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":51,"thread_start_us":116,"threads_started":1}
I20260620 18:46:53.785478 28912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117639d4110a4b63b058bb75ca50a5ff" candidate_uuid: "dbcf11af0a6241389eef679e4043af1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09bb0b6abab042628738c05c0a8c1155" is_pre_election: true
I20260620 18:46:53.785658 28912 raft_consensus.cc:2454] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcf11af0a6241389eef679e4043af1f in term 0.
I20260620 18:46:53.785907 28712 leader_election.cc:302] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09bb0b6abab042628738c05c0a8c1155, dbcf11af0a6241389eef679e4043af1f; no voters:
I20260620 18:46:53.785962 28625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117639d4110a4b63b058bb75ca50a5ff" candidate_uuid: "dbcf11af0a6241389eef679e4043af1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef69a16900441a69288255e9dd34e78" is_pre_election: true
I20260620 18:46:53.786077 28996 raft_consensus.cc:2781] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:46:53.786110 28625 raft_consensus.cc:2454] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbcf11af0a6241389eef679e4043af1f in term 0.
I20260620 18:46:53.786125 28996 raft_consensus.cc:482] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:46:53.786150 28996 raft_consensus.cc:3037] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:53.787097 28996 raft_consensus.cc:504] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 } } peers { permanent_uuid: "dbcf11af0a6241389eef679e4043af1f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44391 } } peers { permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 } }
I20260620 18:46:53.787243 28996 leader_election.cc:288] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [CANDIDATE]: Term 1 election: Requested vote from peers 7ef69a16900441a69288255e9dd34e78 (127.15.77.65:41137), 09bb0b6abab042628738c05c0a8c1155 (127.15.77.67:38605)
I20260620 18:46:53.787479 28625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117639d4110a4b63b058bb75ca50a5ff" candidate_uuid: "dbcf11af0a6241389eef679e4043af1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef69a16900441a69288255e9dd34e78"
I20260620 18:46:53.787565 28625 raft_consensus.cc:3037] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:53.788489 28625 raft_consensus.cc:2454] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcf11af0a6241389eef679e4043af1f in term 1.
I20260620 18:46:53.788789 28711 leader_election.cc:302] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ef69a16900441a69288255e9dd34e78, dbcf11af0a6241389eef679e4043af1f; no voters:
I20260620 18:46:53.788863 28912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "117639d4110a4b63b058bb75ca50a5ff" candidate_uuid: "dbcf11af0a6241389eef679e4043af1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09bb0b6abab042628738c05c0a8c1155"
I20260620 18:46:53.788968 28912 raft_consensus.cc:3037] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:53.789862 28912 raft_consensus.cc:2454] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbcf11af0a6241389eef679e4043af1f in term 1.
I20260620 18:46:53.790056 28996 raft_consensus.cc:2781] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:46:53.790249 28996 raft_consensus.cc:686] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 1 LEADER]: Becoming Leader. State: Replica: dbcf11af0a6241389eef679e4043af1f, State: Running, Role: LEADER
I20260620 18:46:53.790376 28996 consensus_queue.cc:237] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 } } peers { permanent_uuid: "dbcf11af0a6241389eef679e4043af1f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44391 } } peers { permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 } }
I20260620 18:46:53.791288 28491 catalog_manager.cc:5495] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f reported cstate change: term changed from 0 to 1, leader changed from <none> to dbcf11af0a6241389eef679e4043af1f (127.15.77.66). New cstate: current_term: 1 leader_uuid: "dbcf11af0a6241389eef679e4043af1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbcf11af0a6241389eef679e4043af1f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 } health_report { overall_health: UNKNOWN } } }
I20260620 18:46:53.802883 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.799568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.15.77.66:44391)
0620 18:46:53.799614 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:53.799616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:53.799625 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:46:53.799673 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:53.799673 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:53.799690 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:53.799754 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:46:53.799923 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.800660 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.801893 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:53.802204 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:53.802283 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:53.802489 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:46:53.802628 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:46:53.802723 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:53.802773 (+ 50us) server_negotiation.cc:299] Negotiation successful
0620 18:46:53.802796 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:46:53.805253 28912 raft_consensus.cc:1261] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155 [term 1 FOLLOWER]: Refusing update from remote peer dbcf11af0a6241389eef679e4043af1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:46:53.805291 28625 raft_consensus.cc:1261] T 117639d4110a4b63b058bb75ca50a5ff P 7ef69a16900441a69288255e9dd34e78 [term 1 FOLLOWER]: Refusing update from remote peer dbcf11af0a6241389eef679e4043af1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:46:53.805660 29002 consensus_queue.cc:1035] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ef69a16900441a69288255e9dd34e78" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 41137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:46:53.805841 29002 consensus_queue.cc:1035] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "09bb0b6abab042628738c05c0a8c1155" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 38605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:46:53.809074 29005 mvcc.cc:205] Tried to move back new op lower bound from 7298995051743342592 to 7298995051686043648. Current Snapshot: MvccSnapshot[applied={T|T < 7298995051743342592}]
I20260620 18:46:53.809954 29006 mvcc.cc:205] Tried to move back new op lower bound from 7298995051743342592 to 7298995051686043648. Current Snapshot: MvccSnapshot[applied={T|T < 7298995051743342592}]
I20260620 18:46:53.810120 29007 mvcc.cc:205] Tried to move back new op lower bound from 7298995051743342592 to 7298995051686043648. Current Snapshot: MvccSnapshot[applied={T|T < 7298995051743342592}]
I20260620 18:46:53.813529 28492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:46:53.813651 28492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:46:53.814543 28492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 210cf47aa91f4c6fbfc6e5a8a4bacbde: Sending DeleteTablet for 3 replicas of tablet 117639d4110a4b63b058bb75ca50a5ff
I20260620 18:46:53.815012 28758 tablet_service.cc:1514] Processing DeleteTablet for tablet 117639d4110a4b63b058bb75ca50a5ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:46:53 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:49104
I20260620 18:46:53.815044 28610 tablet_service.cc:1514] Processing DeleteTablet for tablet 117639d4110a4b63b058bb75ca50a5ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:46:53 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:37714
I20260620 18:46:53.815253 28892 tablet_service.cc:1514] Processing DeleteTablet for tablet 117639d4110a4b63b058bb75ca50a5ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:46:53 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:55414
I20260620 18:46:53.815352 29014 tablet_replica.cc:330] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: stopping tablet replica
I20260620 18:46:53.815481 29014 raft_consensus.cc:2229] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:46:53.815631 29014 raft_consensus.cc:2258] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:46:53.815913 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 28530
I20260620 18:46:53.816816 29014 ts_tablet_manager.cc:1905] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:46:53.818641 29014 ts_tablet_manager.cc:1918] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:46:53.818714 29014 log.cc:1196] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/117639d4110a4b63b058bb75ca50a5ff
I20260620 18:46:53.819015 29014 ts_tablet_manager.cc:1939] T 117639d4110a4b63b058bb75ca50a5ff P dbcf11af0a6241389eef679e4043af1f: Deleting consensus metadata
I20260620 18:46:53.819612 28477 catalog_manager.cc:4841] TS dbcf11af0a6241389eef679e4043af1f (127.15.77.66:44391): tablet 117639d4110a4b63b058bb75ca50a5ff (table test-table [id=13461b67cab54303b09efe8bd3f1b922]) successfully deleted
I20260620 18:46:53.821928 29016 tablet_replica.cc:330] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: stopping tablet replica
I20260620 18:46:53.822082 29016 raft_consensus.cc:2229] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:46:53.822240 29016 raft_consensus.cc:2258] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:46:53.822875 29016 ts_tablet_manager.cc:1905] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:46:53.824568 29016 ts_tablet_manager.cc:1918] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:46:53.824700 29016 log.cc:1196] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/117639d4110a4b63b058bb75ca50a5ff
W20260620 18:46:53.824805 28476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:41137 (error 108)
I20260620 18:46:53.825121 29016 ts_tablet_manager.cc:1939] T 117639d4110a4b63b058bb75ca50a5ff P 09bb0b6abab042628738c05c0a8c1155: Deleting consensus metadata
I20260620 18:46:53.825466 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 28694
I20260620 18:46:53.826063 28477 catalog_manager.cc:4841] TS 09bb0b6abab042628738c05c0a8c1155 (127.15.77.67:38605): tablet 117639d4110a4b63b058bb75ca50a5ff (table test-table [id=13461b67cab54303b09efe8bd3f1b922]) successfully deleted
I20260620 18:46:53.832912 28988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:53.832746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:41137 (local address 127.0.0.1:37730)
0620 18:46:53.832815 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:53.832868 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:41137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 18:46:53.833168 28476 catalog_manager.cc:4584] TS 7ef69a16900441a69288255e9dd34e78 (127.15.77.65:41137): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 117639d4110a4b63b058bb75ca50a5ff: Network error: Client connection negotiation failed: client connection to 127.15.77.65:41137: connect: Connection refused (error 111)
I20260620 18:46:53.834152 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 28827
I20260620 18:46:53.841409 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 28459
I20260620 18:46:53.847703 15669 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 18:46:54.182236 15669 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 18:46:54.182320 15669 mini_ranger.cc:66] Stopping Ranger...
I20260620 18:46:54.527292 15669 mini_ranger.cc:68] Stopped Ranger
2026-06-20 18:46:54.527 UTC [27526] LOG: received smart shutdown request
2026-06-20 18:46:54.531 UTC [27526] LOG: background worker "logical replication launcher" (PID 27535) exited with exit code 1
2026-06-20 18:46:54.531 UTC [27530] LOG: shutting down
2026-06-20 18:46:54.701 UTC [27526] LOG: database system is shut down
2026-06-20T18:46:54Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48470 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:46:54 dist-test-slave-tvl9 krb5kdc[29026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:46:54 dist-test-slave-tvl9 krb5kdc[29026](info): set up 2 sockets
Jun 20 18:46:54 dist-test-slave-tvl9 krb5kdc[29026](info): commencing operation
krb5kdc: starting...
W20260620 18:46:55.859457 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:46:55 dist-test-slave-tvl9 krb5kdc[29026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:46:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:46:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:46:55.978824 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:33893
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:46641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:33893,127.15.77.125:35687,127.15.77.124:39865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:56.062356 29039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:56.062666 29039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:56.062709 29039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:56.064021 29039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:46:56.064105 29039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:56.064134 29039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:56.064167 29039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:46:56.064191 29039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:46:56.065764 29039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:46641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:33893,127.15.77.125:35687,127.15.77.124:39865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:33893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:56.066071 29039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:56.066315 29039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:56.071300 29044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:56.071434 29047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:56.071333 29045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:56.071477 29039 server_base.cc:1034] running on GCE node
I20260620 18:46:56.071895 29039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:56.072235 29039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:56.073406 29039 hybrid_clock.cc:648] HybridClock initialized: now 1781981216073387 us; error 40 us; skew 500 ppm
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:56.080423 29039 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:46:56.080969 29039 webserver.cc:460] Webserver started at http://127.15.77.126:38285/ using document root <none> and password file <none>
I20260620 18:46:56.081183 29039 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:56.081225 29039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:56.081321 29039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:46:56.082394 29039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "765713a9d0f44087851e49529e515541"
format_stamp: "Formatted at 2026-06-20 18:46:56 on dist-test-slave-tvl9"
server_key: "eb1c55af5ade114a3ddcc45d2dbdf39e"
server_key_iv: "f6861616bea23e993b0618755a6de5a1"
server_key_version: "encryptionkey@0"
I20260620 18:46:56.082713 29039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "765713a9d0f44087851e49529e515541"
format_stamp: "Formatted at 2026-06-20 18:46:56 on dist-test-slave-tvl9"
server_key: "eb1c55af5ade114a3ddcc45d2dbdf39e"
server_key_iv: "f6861616bea23e993b0618755a6de5a1"
server_key_version: "encryptionkey@0"
I20260620 18:46:56.084194 29039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:46:56.084960 29054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:56.085175 29039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:46:56.085269 29039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "765713a9d0f44087851e49529e515541"
format_stamp: "Formatted at 2026-06-20 18:46:56 on dist-test-slave-tvl9"
server_key: "eb1c55af5ade114a3ddcc45d2dbdf39e"
server_key_iv: "f6861616bea23e993b0618755a6de5a1"
server_key_version: "encryptionkey@0"
I20260620 18:46:56.085331 29039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:56.105027 29039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:56.107261 29039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:56.107481 29039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:56.111817 29039 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:33893
I20260620 18:46:56.111884 29106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:33893 every 8 connection(s)
I20260620 18:46:56.112278 29039 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:46:56.113226 29107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:56.113754 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 29039
I20260620 18:46:56.113833 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:46:56.114056 15669 external_mini_cluster.cc:1428] Setting key c1367f8570f43b6017f6ee770797d9b4
I20260620 18:46:56.115029 29107 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.77.125@KRBTEST.COM; defaulting to no policy
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.125@KRBTEST.COM" created.
I20260620 18:46:56.127141 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.115993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:33893 (local address 127.0.0.1:59168)
0620 18:46:56.116204 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.116244 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:56.116725 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:56.117060 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:56.117062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:56.117338 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:56.117504 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.117508 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.119131 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.119132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.119934 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.119937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.120674 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.120675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.120742 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.121343 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:56.121352 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:56.123014 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:56.125374 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.125382 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.125385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.125691 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.125944 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.125945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.125946 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.126065 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.126424 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:56.126432 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:56.126514 (+ 82us) client_negotiation.cc:769] Sending connection context
0620 18:46:56.126638 (+ 124us) client_negotiation.cc:240] Negotiation successful
0620 18:46:56.126937 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":135,"thread_start_us":95,"threads_started":1}
I20260620 18:46:56.127151 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.116025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.15.77.126:33893)
0620 18:46:56.116207 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.116218 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.116332 (+ 114us) server_negotiation.cc:407] Connection header received
0620 18:46:56.116896 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.116898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.116930 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.117037 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.117613 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.119011 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.120024 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.120572 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.120700 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.123113 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.123121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.123121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.123128 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.125277 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.125787 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.125790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.125791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.125878 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.126132 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.126132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.126133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.126353 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.126417 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.126692 (+ 275us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.126874 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260620 18:46:56.128077 29107 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:46:56.128456 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.128328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35687 (local address 127.0.0.1:60054)
0620 18:46:56.128381 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.128427 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 18:46:56.128582 29057 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35687: connect: Connection refused (error 111)
I20260620 18:46:56.129010 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.128913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35687 (local address 127.0.0.1:60068)
0620 18:46:56.128959 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.128987 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 18:46:56.129163 29107 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.125:35687: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35687: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab.
Entry for principal kudu/127.15.77.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab.
I20260620 18:46:56.143013 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.125:35687
--webserver_interface=127.15.77.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab
--principal=kudu/127.15.77.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:46641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:33893,127.15.77.125:35687,127.15.77.124:39865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:46:56.178424 29107 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } attempt: 1
I20260620 18:46:56.178951 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.178809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35687 (local address 127.0.0.1:60070)
0620 18:46:56.178878 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.178921 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 18:46:56.179427 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.179294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35687 (local address 127.0.0.1:60078)
0620 18:46:56.179363 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.179392 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260620 18:46:56.179541 29107 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.125:35687: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35687: connect: Connection refused (error 111)
W20260620 18:46:56.228729 29114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:56.228963 29114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:56.228988 29114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:56.230232 29114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:46:56.230281 29114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:56.230296 29114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:56.230309 29114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:46:56.230372 29114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:46:56.232028 29114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:46641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:33893,127.15.77.125:35687,127.15.77.124:39865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab
--principal=kudu/127.15.77.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.125:35687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.77.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:56.232342 29114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:56.232595 29114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:56.237989 29119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:56.238052 29120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:56.238178 29122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:56.238021 29114 server_base.cc:1034] running on GCE node
I20260620 18:46:56.238595 29114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:56.238922 29114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:56.240091 29114 hybrid_clock.cc:648] HybridClock initialized: now 1781981216240065 us; error 67 us; skew 500 ppm
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:56.242825 29114 init.cc:377] Logged in from keytab as kudu/127.15.77.125@KRBTEST.COM (short username kudu)
I20260620 18:46:56.243462 29114 webserver.cc:460] Webserver started at http://127.15.77.125:45009/ using document root <none> and password file <none>
I20260620 18:46:56.243737 29114 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:56.243784 29114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:56.243934 29114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:46:56.245036 29114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/data/instance:
uuid: "37742d33c8eb43979ddf4de6ab2f2fe6"
format_stamp: "Formatted at 2026-06-20 18:46:56 on dist-test-slave-tvl9"
server_key: "29a41dcb759297d66ba76f4bc3a55ecf"
server_key_iv: "f27c38c5b03fe2ab3aa34fee4e309c36"
server_key_version: "encryptionkey@0"
I20260620 18:46:56.245410 29114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/wal/instance:
uuid: "37742d33c8eb43979ddf4de6ab2f2fe6"
format_stamp: "Formatted at 2026-06-20 18:46:56 on dist-test-slave-tvl9"
server_key: "29a41dcb759297d66ba76f4bc3a55ecf"
server_key_iv: "f27c38c5b03fe2ab3aa34fee4e309c36"
server_key_version: "encryptionkey@0"
I20260620 18:46:56.246783 29114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:46:56.247607 29129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:56.247682 29107 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } attempt: 2
I20260620 18:46:56.247848 29114 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:46:56.247941 29114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/wal
uuid: "37742d33c8eb43979ddf4de6ab2f2fe6"
format_stamp: "Formatted at 2026-06-20 18:46:56 on dist-test-slave-tvl9"
server_key: "29a41dcb759297d66ba76f4bc3a55ecf"
server_key_iv: "f27c38c5b03fe2ab3aa34fee4e309c36"
server_key_version: "encryptionkey@0"
I20260620 18:46:56.248037 29114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:56.248193 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.248036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35687 (local address 127.0.0.1:60082)
0620 18:46:56.248114 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.248146 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260620 18:46:56.248543 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.248449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35687 (local address 127.0.0.1:60094)
0620 18:46:56.248502 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.248527 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 18:46:56.248663 29107 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.125:35687: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35687: connect: Connection refused (error 111)
I20260620 18:46:56.258836 29114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:56.260866 29114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:56.261035 29114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:56.265494 29114 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.125:35687
I20260620 18:46:56.265547 29181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.125:35687 every 8 connection(s)
I20260620 18:46:56.265949 29114 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
I20260620 18:46:56.266878 29182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:56.268507 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 29114
I20260620 18:46:56.268603 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-1/wal/instance
I20260620 18:46:56.268778 15669 external_mini_cluster.cc:1428] Setting key 038e37e15fb8bdfc418d4561e98f74e5
I20260620 18:46:56.268841 29182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.124@KRBTEST.COM" created.
I20260620 18:46:56.280166 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.269762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59180 (local address 127.15.77.126:33893)
0620 18:46:56.269836 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.269839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.270111 (+ 272us) server_negotiation.cc:407] Connection header received
0620 18:46:56.270733 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.270735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.270752 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.270826 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.271647 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.272440 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.273492 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.273882 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.273959 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.277121 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.277124 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.277124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.277134 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.278522 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.278908 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.278910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.278910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.279000 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.279231 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.279232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.279232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.279433 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.279536 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.279918 (+ 382us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.280051 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 18:46:56.280236 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.269744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:33893 (local address 127.0.0.1:59180)
0620 18:46:56.269955 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.270005 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:56.270620 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:56.270894 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:56.270897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:56.271207 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:56.271555 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.271561 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.272539 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.272540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.273395 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.273398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.273979 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.273980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.274151 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.275013 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:56.275025 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:56.277012 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:56.278586 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.278594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.278596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.278837 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.279061 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.279061 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.279062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.279173 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.279543 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:56.279551 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:56.279744 (+ 193us) client_negotiation.cc:769] Sending connection context
0620 18:46:56.279868 (+ 124us) client_negotiation.cc:240] Negotiation successful
0620 18:46:56.280057 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":108,"threads_started":1}
I20260620 18:46:56.280887 29182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
I20260620 18:46:56.288800 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.281174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35687 (local address 127.0.0.1:60100)
0620 18:46:56.281230 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.281241 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:56.281307 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:56.281542 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:56.281543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:56.281691 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:56.281738 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.281739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.282591 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.282592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.283087 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.283089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.283718 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.283719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.283787 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.284246 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:56.284252 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:56.285659 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:56.287606 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.287607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.287607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.287793 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.288043 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.288044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.288044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.288097 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.288494 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:56.288495 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:56.288533 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 18:46:56.288586 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 18:46:56.288664 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 18:46:56.288969 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.281229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60100 (local address 127.15.77.125:35687)
0620 18:46:56.281415 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.281419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.281433 (+ 14us) server_negotiation.cc:407] Connection header received
0620 18:46:56.281448 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.281450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.281460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.281541 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.281817 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.282499 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.283192 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.283622 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.283719 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.285757 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.285765 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.285766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.285773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.287508 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.287885 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.287890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.287891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.287966 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.288188 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.288189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.288189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.288423 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.288495 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.288703 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.288771 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
I20260620 18:46:56.289533 29182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:46:56.289922 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.289806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47678)
0620 18:46:56.289858 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.289902 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 18:46:56.290071 29130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
I20260620 18:46:56.290500 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.290402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47680)
0620 18:46:56.290453 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.290477 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 18:46:56.290609 29182 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:39865: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab.
Entry for principal kudu/127.15.77.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab.
I20260620 18:46:56.296495 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.124:39865
--webserver_interface=127.15.77.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab
--principal=kudu/127.15.77.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:46641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:33893,127.15.77.125:35687,127.15.77.124:39865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:46:56.339879 29107 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } attempt: 3
I20260620 18:46:56.339871 29182 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } attempt: 1
I20260620 18:46:56.340443 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.340285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47690)
0620 18:46:56.340364 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.340406 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260620 18:46:56.340818 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.340715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47702)
0620 18:46:56.340768 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.340797 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 18:46:56.340926 29182 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:39865: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
I20260620 18:46:56.347179 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.340243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35687 (local address 127.0.0.1:60108)
0620 18:46:56.340299 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.340314 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:56.340368 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:56.340551 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:56.340552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:56.340773 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:56.340878 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.340881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.341577 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.341578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.342079 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.342080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.342488 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.342490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.342567 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.343068 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:56.343073 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:56.344499 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:56.346100 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.346102 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.346102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.346293 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.346499 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.346500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.346500 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.346566 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.346826 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:56.346827 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:56.346878 (+ 51us) client_negotiation.cc:769] Sending connection context
0620 18:46:56.346966 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 18:46:56.347073 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260620 18:46:56.347173 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.340329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60108 (local address 127.15.77.125:35687)
0620 18:46:56.340376 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.340378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.340419 (+ 41us) server_negotiation.cc:407] Connection header received
0620 18:46:56.340447 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.340448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.340458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.340565 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.340974 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.341483 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.342177 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.342399 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.342477 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.344596 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.344598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.344598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.344607 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.346005 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.346368 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.346370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.346370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.346429 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.346629 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.346630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.346631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.346756 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.346834 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.346991 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.347073 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 18:46:56.347595 29107 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:46:56.347991 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.347869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47710)
0620 18:46:56.347922 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.347960 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 18:46:56.348287 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.348214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47726)
0620 18:46:56.348243 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.348268 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260620 18:46:56.348380 29107 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:39865: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
I20260620 18:46:56.379520 29107 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } attempt: 1
I20260620 18:46:56.380064 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.379907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47732)
0620 18:46:56.379980 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.380030 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 18:46:56.380581 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.380403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47734)
0620 18:46:56.380521 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.380556 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260620 18:46:56.380750 29107 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:39865: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
W20260620 18:46:56.384438 29189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:56.384692 29189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:56.384745 29189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:56.385999 29189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:46:56.386075 29189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:56.386111 29189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:56.386138 29189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:46:56.386165 29189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:46:56.388006 29189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:46641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:33893,127.15.77.125:35687,127.15.77.124:39865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab
--principal=kudu/127.15.77.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.124:39865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.77.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:56.388329 29189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:56.388592 29189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:56.393827 29195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:56.393854 29194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:56.393831 29197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:56.394084 29189 server_base.cc:1034] running on GCE node
I20260620 18:46:56.394333 29189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:56.394647 29189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:56.395818 29189 hybrid_clock.cc:648] HybridClock initialized: now 1781981216395806 us; error 46 us; skew 500 ppm
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:56.398275 29189 init.cc:377] Logged in from keytab as kudu/127.15.77.124@KRBTEST.COM (short username kudu)
I20260620 18:46:56.398733 29189 webserver.cc:460] Webserver started at http://127.15.77.124:38427/ using document root <none> and password file <none>
I20260620 18:46:56.398967 29189 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:56.399042 29189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:56.399199 29189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:46:56.400352 29189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/data/instance:
uuid: "11509b3b47bb4767b26ba09745440bd3"
format_stamp: "Formatted at 2026-06-20 18:46:56 on dist-test-slave-tvl9"
server_key: "bf62512d91333317296d159276b0f6d3"
server_key_iv: "54e779344f7bc9611a89551e1a48b2d6"
server_key_version: "encryptionkey@0"
I20260620 18:46:56.400724 29189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/wal/instance:
uuid: "11509b3b47bb4767b26ba09745440bd3"
format_stamp: "Formatted at 2026-06-20 18:46:56 on dist-test-slave-tvl9"
server_key: "bf62512d91333317296d159276b0f6d3"
server_key_iv: "54e779344f7bc9611a89551e1a48b2d6"
server_key_version: "encryptionkey@0"
I20260620 18:46:56.402076 29189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:46:56.402876 29204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:56.403046 29189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:46:56.403149 29189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/wal
uuid: "11509b3b47bb4767b26ba09745440bd3"
format_stamp: "Formatted at 2026-06-20 18:46:56 on dist-test-slave-tvl9"
server_key: "bf62512d91333317296d159276b0f6d3"
server_key_iv: "54e779344f7bc9611a89551e1a48b2d6"
server_key_version: "encryptionkey@0"
I20260620 18:46:56.403234 29189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:56.409056 29182 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } attempt: 2
I20260620 18:46:56.409469 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.409343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47750)
0620 18:46:56.409403 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.409440 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 18:46:56.409808 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.409712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47758)
0620 18:46:56.409758 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.409782 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 18:46:56.409895 29182 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:39865: Network error: Client connection negotiation failed: client connection to 127.15.77.124:39865: connect: Connection refused (error 111)
I20260620 18:46:56.412798 29189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:56.414811 29189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:56.414984 29189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:56.419217 29189 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.124:39865
I20260620 18:46:56.419265 29256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.124:39865 every 8 connection(s)
I20260620 18:46:56.419727 29189 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
I20260620 18:46:56.420652 29257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:56.421751 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 29189
I20260620 18:46:56.421885 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781981032911509-15669-0/minicluster-data/master-2/wal/instance
I20260620 18:46:56.422079 15669 external_mini_cluster.cc:1428] Setting key 95487b07bb19193d03473fb85c9adcf9
I20260620 18:46:56.422552 29257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
I20260620 18:46:56.433022 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.423348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59194 (local address 127.15.77.126:33893)
0620 18:46:56.423407 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.423409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.423628 (+ 219us) server_negotiation.cc:407] Connection header received
0620 18:46:56.423656 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.423658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.423669 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.423718 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.424562 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.425340 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.426691 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.426992 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.427068 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.429567 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.429569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.429569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.429579 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.431285 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.431873 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.431875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.431876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.431939 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.432237 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.432237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.432237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.432399 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.432466 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.432829 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.432917 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 18:46:56.433240 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.423260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:33893 (local address 127.0.0.1:59194)
0620 18:46:56.423471 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.423509 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:56.423590 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:56.423740 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:56.423743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:56.424206 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:56.424467 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.424473 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.425450 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.425451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.426587 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.426590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.427086 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.427087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.427189 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.427750 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:56.427779 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:56.429471 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:56.431412 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.431419 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.431421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.431801 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.432005 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.432006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.432006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.432167 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.432469 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:56.432476 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:56.432654 (+ 178us) client_negotiation.cc:769] Sending connection context
0620 18:46:56.432740 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 18:46:56.432924 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":111,"threads_started":1}
I20260620 18:46:56.433243 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.422670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47764 (local address 127.15.77.124:39865)
0620 18:46:56.422886 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.422896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.422931 (+ 35us) server_negotiation.cc:407] Connection header received
0620 18:46:56.423472 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.423480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.423531 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.423668 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.424504 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.425610 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.426691 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.427189 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.427240 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.430116 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.430125 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.430125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.430133 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.431905 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.432305 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.432308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.432309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.432366 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.432586 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.432587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.432587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.432713 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.432872 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.432959 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.433027 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
I20260620 18:46:56.434082 29257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
I20260620 18:46:56.441787 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.434170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59198 (local address 127.15.77.126:33893)
0620 18:46:56.434240 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.434242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.434268 (+ 26us) server_negotiation.cc:407] Connection header received
0620 18:46:56.434287 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.434288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.434299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.434351 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.434816 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.435256 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.436171 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.436449 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.436520 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.438897 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.438899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.438899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.438907 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.440599 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.440959 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.440960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.440960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.441008 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.441204 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.441205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.441205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.441357 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.441551 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.441605 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.441686 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 18:46:56.442121 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.434336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35687 (local address 127.0.0.1:60122)
0620 18:46:56.434825 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.434834 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:56.434892 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:56.435025 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:56.435025 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:56.435162 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:56.435198 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.435200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.436112 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.436113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.436757 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.436759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.437178 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.437179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.437256 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.437577 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:56.437582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:56.439248 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:56.440889 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.440890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.440891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.441118 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.441342 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.441343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.441343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.441407 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.441775 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:56.441776 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:56.441821 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 18:46:56.441938 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 18:46:56.442038 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475}
I20260620 18:46:56.442446 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.434412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.15.77.125:35687)
0620 18:46:56.434455 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.434457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.434887 (+ 430us) server_negotiation.cc:407] Connection header received
0620 18:46:56.434954 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.434955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.434966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.435021 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.435287 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.436032 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.436831 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.437101 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.437166 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.439359 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.439362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.439363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.439373 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.440817 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.441205 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.441206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.441207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.441275 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.441480 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.441481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.441481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.441617 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.441688 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.441994 (+ 306us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.442354 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:46:56.442906 29257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
I20260620 18:46:56.449551 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.443176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47772)
0620 18:46:56.443226 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.443235 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:56.443270 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:56.443438 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:56.443439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:56.443594 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:56.443625 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.443627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.444248 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.444249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.444704 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.444706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.445096 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.445097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.445158 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.445694 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:56.445698 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:56.446952 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:56.448551 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.448552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.448553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.448714 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.448922 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.448922 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.448923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.448974 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.449225 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:56.449225 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:56.449263 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 18:46:56.449333 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 18:46:56.449432 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 18:46:56.449595 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.443238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47772 (local address 127.15.77.124:39865)
0620 18:46:56.443279 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.443281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.443289 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:46:56.443352 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.443352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.443363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.443436 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.443697 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.444169 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.444815 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.444995 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.445077 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.447034 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.447036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.447037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.447044 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.448458 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.448798 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.448799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.448800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.448848 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.449045 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.449046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.449046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.449159 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.449232 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.449358 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.449483 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":22}
I20260620 18:46:56.450838 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.442646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.15.77.125:35687)
0620 18:46:56.442726 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.442728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.442736 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:46:56.442753 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.442755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.442764 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.442873 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.443495 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.444069 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.444754 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.444978 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.445026 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.447788 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.447789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.447789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.447795 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.449200 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.449560 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.449561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.449561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.450010 (+ 449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.450298 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.450299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.450299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.450407 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.450615 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.450654 (+ 39us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.450727 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65,"thread_start_us":102,"threads_started":1}
I20260620 18:46:56.451275 29257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3: Bootstrap starting.
I20260620 18:46:56.452471 29257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:56.452854 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:46:56.452930 29257 log.cc:824] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:56.453660 29257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3: No bootstrap required, opened a new log
I20260620 18:46:56.455660 29257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } }
I20260620 18:46:56.455927 29107 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } attempt: 2
I20260620 18:46:56.456101 29257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:56.456219 29257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11509b3b47bb4767b26ba09745440bd3, State: Initialized, Role: FOLLOWER
I20260620 18:46:56.456429 29257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } }
I20260620 18:46:56.456820 29257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:46:56.459105 29272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } } }
I20260620 18:46:56.459270 29272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:46:56.458954 29189 master_runner.cc:186] Error getting master registration for 127.15.77.126:33893: OK, instance_id { permanent_uuid: "765713a9d0f44087851e49529e515541" instance_seqno: 1781981216110555 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 18:46:56.462095 29189 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 18:46:56.458973 29283 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:46:56.462846 29283 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
I20260620 18:46:56.464756 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.454554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60150 (local address 127.15.77.125:35687)
0620 18:46:56.454587 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.454588 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.455432 (+ 844us) server_negotiation.cc:407] Connection header received
0620 18:46:56.455525 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.455526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.455535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.455565 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.456694 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.457376 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.458296 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.458567 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.458624 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.460728 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.460729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.460730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.460737 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.462578 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.462919 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.462920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.462920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.462977 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.463678 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.463679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.463679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.463810 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.464162 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.464572 (+ 410us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.464657 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 18:46:56.465938 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.454241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59212 (local address 127.15.77.126:33893)
0620 18:46:56.454287 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.454289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.454408 (+ 119us) server_negotiation.cc:407] Connection header received
0620 18:46:56.454519 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.454520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.454532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.454564 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.455182 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.455935 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.456955 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.457239 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.457286 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.463068 (+ 5782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.463069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.463070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.463077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.464688 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.465066 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.465067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.465067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.465121 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.465343 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.465344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.465344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.465466 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.465536 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.465755 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.465833 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:46:56.466337 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.454489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.15.77.124:39865)
0620 18:46:56.454525 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.454527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.455250 (+ 723us) server_negotiation.cc:407] Connection header received
0620 18:46:56.455303 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.455304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.455333 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.455382 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.456116 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.456774 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.458028 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.458334 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.458371 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.460157 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.460159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.460159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.460166 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.461916 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.462388 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.462390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.462391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.462838 (+ 447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.465407 (+ 2569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.465408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.465409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.465845 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.465917 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.466159 (+ 242us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.466242 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 18:46:56.469269 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.456351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47794 (local address 127.15.77.124:39865)
0620 18:46:56.458934 (+ 2583us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.458936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.458944 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:46:56.458989 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.458990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.459000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.460449 (+ 1449us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.460664 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.461438 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.462482 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.462832 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.463131 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.465240 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.465241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.465241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.465249 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.468101 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.468495 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.468496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.468497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.468575 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.468811 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.468812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.468812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.468923 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.468975 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.469141 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.469200 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2562,"thread_start_us":67,"threads_started":1}
I20260620 18:46:56.469309 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.456155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47794)
0620 18:46:56.456202 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.456210 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:56.456241 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:56.459443 (+ 3202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:56.459444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:56.459627 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:56.459675 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.459676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.461541 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.461543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.462375 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.462378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.462927 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.462928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.462987 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.463383 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:56.463388 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:56.464955 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:56.468194 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.468195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.468195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.468416 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.468666 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.468667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.468667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.468736 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.468991 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:56.468992 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:56.469036 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 18:46:56.469113 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 18:46:56.469211 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 18:46:56.471220 29107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541: Bootstrap starting.
I20260620 18:46:56.471971 29107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:56.472291 29107 log.cc:824] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:56.473201 29107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541: No bootstrap required, opened a new log
I20260620 18:46:56.475210 29107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } }
I20260620 18:46:56.475394 29107 raft_consensus.cc:374] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:56.475440 29107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 765713a9d0f44087851e49529e515541, State: Initialized, Role: FOLLOWER
I20260620 18:46:56.475586 29107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } }
I20260620 18:46:56.475863 29286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } } }
I20260620 18:46:56.475904 29107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:46:56.475970 29286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 18:46:56.477720 29297 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:46:56.477819 29297 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:46:56.478403 29039 master_runner.cc:186] Error getting master registration for 127.15.77.125:35687: OK, instance_id { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" instance_seqno: 1781981216264231 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 18:46:56.480998 29039 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:46:56.501062 29182 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } attempt: 3
Jun 20 18:46:56 dist-test-slave-tvl9 krb5kdc[29026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981216, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
I20260620 18:46:56.509656 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.501391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:39865 (local address 127.0.0.1:47802)
0620 18:46:56.501445 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 18:46:56.501456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:46:56.501495 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:46:56.501666 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:46:56.501667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:46:56.501866 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:46:56.501920 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.501922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.502920 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.502922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.503765 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:46:56.503768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.504266 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.504267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:46:56.504327 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.504857 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:46:56.504863 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:46:56.506437 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:46:56.508492 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.508493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.508494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.508727 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.508943 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:46:56.508944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:46:56.508944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:46:56.509010 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:46:56.509307 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:46:56.509308 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:46:56.509360 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 18:46:56.509436 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 18:46:56.509537 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 18:46:56.509671 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:56.501478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47802 (local address 127.15.77.124:39865)
0620 18:46:56.501523 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:56.501525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:56.501534 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:46:56.501560 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:56.501561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:56.501571 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:56.501641 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:56.502005 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.502816 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.503848 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:56.504179 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:56.504251 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:56.506541 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:56.506542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:56.506543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:56.506551 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:56.508398 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.508807 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.508808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.508809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.508870 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:56.509079 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:56.509080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:56.509080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:56.509232 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:56.509303 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:56.509489 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 18:46:56.509574 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:46:56.511870 29182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6: Bootstrap starting.
I20260620 18:46:56.512641 29182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:56.512996 29182 log.cc:824] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:56.513933 29182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6: No bootstrap required, opened a new log
I20260620 18:46:56.516124 29182 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } }
I20260620 18:46:56.516278 29182 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:56.516314 29182 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37742d33c8eb43979ddf4de6ab2f2fe6, State: Initialized, Role: FOLLOWER
I20260620 18:46:56.517139 29182 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } }
I20260620 18:46:56.517459 29300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } } }
I20260620 18:46:56.517539 29182 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:46:56.517622 29300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 18:46:56.519526 29311 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:46:56.519634 29311 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:46:56.521598 29114 master_runner.cc:263] No leader master found from master 37742d33c8eb43979ddf4de6ab2f2fe6
I20260620 18:46:56.521644 29114 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:46:56.543164 29272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:46:56.543290 29272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } }
I20260620 18:46:56.543548 29272 leader_election.cc:288] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 765713a9d0f44087851e49529e515541 (127.15.77.126:33893), 37742d33c8eb43979ddf4de6ab2f2fe6 (127.15.77.125:35687)
I20260620 18:46:56.544056 29081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11509b3b47bb4767b26ba09745440bd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "765713a9d0f44087851e49529e515541" is_pre_election: true
I20260620 18:46:56.544231 29081 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11509b3b47bb4767b26ba09745440bd3 in term 0.
I20260620 18:46:56.544453 29156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11509b3b47bb4767b26ba09745440bd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" is_pre_election: true
I20260620 18:46:56.544593 29156 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11509b3b47bb4767b26ba09745440bd3 in term 0.
I20260620 18:46:56.544886 29207 leader_election.cc:302] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11509b3b47bb4767b26ba09745440bd3, 37742d33c8eb43979ddf4de6ab2f2fe6; no voters:
I20260620 18:46:56.545217 29272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:46:56.545277 29272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:46:56.545300 29272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:56.546209 29272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } }
I20260620 18:46:56.546330 29272 leader_election.cc:288] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [CANDIDATE]: Term 1 election: Requested vote from peers 765713a9d0f44087851e49529e515541 (127.15.77.126:33893), 37742d33c8eb43979ddf4de6ab2f2fe6 (127.15.77.125:35687)
I20260620 18:46:56.546720 29081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11509b3b47bb4767b26ba09745440bd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "765713a9d0f44087851e49529e515541"
I20260620 18:46:56.546803 29081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:56.547435 29156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11509b3b47bb4767b26ba09745440bd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6"
I20260620 18:46:56.547524 29156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:56.548456 29156 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11509b3b47bb4767b26ba09745440bd3 in term 1.
I20260620 18:46:56.548707 29207 leader_election.cc:302] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11509b3b47bb4767b26ba09745440bd3, 37742d33c8eb43979ddf4de6ab2f2fe6; no voters:
I20260620 18:46:56.549211 29081 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11509b3b47bb4767b26ba09745440bd3 in term 1.
I20260620 18:46:56.549454 29272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:46:56.549652 29272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [term 1 LEADER]: Becoming Leader. State: Replica: 11509b3b47bb4767b26ba09745440bd3, State: Running, Role: LEADER
I20260620 18:46:56.549752 29272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } }
I20260620 18:46:56.550139 29272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11509b3b47bb4767b26ba09745440bd3. Latest consensus state: current_term: 1 leader_uuid: "11509b3b47bb4767b26ba09745440bd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } } }
I20260620 18:46:56.550209 29272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:56.550536 29315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:46:56.550889 29315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:46:56.553130 29156 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [term 1 FOLLOWER]: Refusing update from remote peer 11509b3b47bb4767b26ba09745440bd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:46:56.553493 29272 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:46:56.554940 29300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11509b3b47bb4767b26ba09745440bd3. Latest consensus state: current_term: 1 leader_uuid: "11509b3b47bb4767b26ba09745440bd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } } }
I20260620 18:46:56.555028 29300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:46:56.556383 29081 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [term 1 FOLLOWER]: Refusing update from remote peer 11509b3b47bb4767b26ba09745440bd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:46:56.557196 29313 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:46:56.557739 29272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11509b3b47bb4767b26ba09745440bd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } } }
I20260620 18:46:56.557822 29272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:56.558194 29316 mvcc.cc:205] Tried to move back new op lower bound from 7298995062999035904 to 7298995062988419072. Current Snapshot: MvccSnapshot[applied={T|T < 7298995062999035904}]
I20260620 18:46:56.559080 29286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11509b3b47bb4767b26ba09745440bd3. Latest consensus state: current_term: 1 leader_uuid: "11509b3b47bb4767b26ba09745440bd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } } }
I20260620 18:46:56.559162 29315 catalog_manager.cc:1347] Generated new cluster ID: 677643129e6c424e89cc87b08597cc17
I20260620 18:46:56.559194 29286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:46:56.559208 29315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:46:56.559242 29286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11509b3b47bb4767b26ba09745440bd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } } }
I20260620 18:46:56.559284 29286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:46:56.563378 29300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11509b3b47bb4767b26ba09745440bd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } } }
I20260620 18:46:56.563488 29300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:46:56.567481 29317 mvcc.cc:205] Tried to move back new op lower bound from 7298995062999035904 to 7298995062988419072. Current Snapshot: MvccSnapshot[applied={T|T < 7298995062999035904}]
I20260620 18:46:56.568651 29321 mvcc.cc:205] Tried to move back new op lower bound from 7298995062999035904 to 7298995062988419072. Current Snapshot: MvccSnapshot[applied={T|T < 7298995062999035904}]
I20260620 18:46:56.570299 29272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11509b3b47bb4767b26ba09745440bd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765713a9d0f44087851e49529e515541" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33893 } } peers { permanent_uuid: "37742d33c8eb43979ddf4de6ab2f2fe6" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35687 } } peers { permanent_uuid: "11509b3b47bb4767b26ba09745440bd3" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 39865 } } }
I20260620 18:46:56.570398 29272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3 [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:56.572876 29315 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:46:56.573647 29315 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:46:56.586448 29315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11509b3b47bb4767b26ba09745440bd3: Generated new TSK 0
I20260620 18:46:56.587718 29315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:46:57.478317 29297 catalog_manager.cc:1259] Loaded cluster ID: 677643129e6c424e89cc87b08597cc17
I20260620 18:46:57.478400 29297 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541: loading cluster ID for follower catalog manager: success
I20260620 18:46:57.479266 29297 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541: acquiring CA information for follower catalog manager: success
I20260620 18:46:57.479655 29297 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 765713a9d0f44087851e49529e515541: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:46:57.520147 29311 catalog_manager.cc:1259] Loaded cluster ID: 677643129e6c424e89cc87b08597cc17
I20260620 18:46:57.520254 29311 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6: loading cluster ID for follower catalog manager: success
I20260620 18:46:57.521064 29311 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6: acquiring CA information for follower catalog manager: success
I20260620 18:46:57.521430 29311 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 37742d33c8eb43979ddf4de6ab2f2fe6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:46:57.652838 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 29039
I20260620 18:46:57.658736 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 29114
I20260620 18:46:57.663810 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 29189
2026-06-20T18:46:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2851 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:46:57 dist-test-slave-tvl9 krb5kdc[29334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:46:57 dist-test-slave-tvl9 krb5kdc[29334](info): set up 2 sockets
Jun 20 18:46:57 dist-test-slave-tvl9 krb5kdc[29334](info): commencing operation
krb5kdc: starting...
W20260620 18:46:58.699301 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:46:58 dist-test-slave-tvl9 krb5kdc[29334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:46:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:46:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:46:58.823196 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34567
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:38313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:34567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:46:58.904424 29347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:46:58.904685 29347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:46:58.904727 29347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:46:58.905913 29347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:46:58.905984 29347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:46:58.906009 29347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:46:58.906029 29347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:46:58.906060 29347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:46:58.907603 29347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:34567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:46:58.907889 29347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:46:58.908188 29347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:46:58.913388 29353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:58.913388 29352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:46:58.913612 29355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:46:58.913837 29347 server_base.cc:1034] running on GCE node
I20260620 18:46:58.914103 29347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:46:58.914410 29347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:46:58.915565 29347 hybrid_clock.cc:648] HybridClock initialized: now 1781981218915555 us; error 29 us; skew 500 ppm
Jun 20 18:46:58 dist-test-slave-tvl9 krb5kdc[29334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981218, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:46:58.918138 29347 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:46:58.918659 29347 webserver.cc:460] Webserver started at http://127.15.77.126:46425/ using document root <none> and password file <none>
I20260620 18:46:58.918900 29347 fs_manager.cc:359] Metadata directory not provided
I20260620 18:46:58.918970 29347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:46:58.919109 29347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:46:58.920323 29347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "1756961096a5436c9478a423a41d4b8f"
format_stamp: "Formatted at 2026-06-20 18:46:58 on dist-test-slave-tvl9"
server_key: "dfe0bed328837533d9fcc616efedf3f4"
server_key_iv: "dc09a390d502d0330bdd026a23c07f71"
server_key_version: "encryptionkey@0"
I20260620 18:46:58.920784 29347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "1756961096a5436c9478a423a41d4b8f"
format_stamp: "Formatted at 2026-06-20 18:46:58 on dist-test-slave-tvl9"
server_key: "dfe0bed328837533d9fcc616efedf3f4"
server_key_iv: "dc09a390d502d0330bdd026a23c07f71"
server_key_version: "encryptionkey@0"
I20260620 18:46:58.922271 29347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:46:58.923163 29362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:46:58.923398 29347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:46:58.923504 29347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "1756961096a5436c9478a423a41d4b8f"
format_stamp: "Formatted at 2026-06-20 18:46:58 on dist-test-slave-tvl9"
server_key: "dfe0bed328837533d9fcc616efedf3f4"
server_key_iv: "dc09a390d502d0330bdd026a23c07f71"
server_key_version: "encryptionkey@0"
I20260620 18:46:58.923593 29347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:46:58.944025 29347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:46:58.946014 29347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:46:58.946193 29347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:46:58.950475 29347 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:34567
I20260620 18:46:58.950516 29414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:34567 every 8 connection(s)
I20260620 18:46:58.950904 29347 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:46:58.951789 29415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:46:58.954140 29415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f: Bootstrap starting.
I20260620 18:46:58.954748 29415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f: Neither blocks nor log segments found. Creating new log.
I20260620 18:46:58.955054 29415 log.cc:824] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f: Log is configured to *not* fsync() on all Append() calls
I20260620 18:46:58.955821 29415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f: No bootstrap required, opened a new log
I20260620 18:46:58.957435 29415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756961096a5436c9478a423a41d4b8f" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34567 } }
I20260620 18:46:58.957576 29415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:46:58.957598 29415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1756961096a5436c9478a423a41d4b8f, State: Initialized, Role: FOLLOWER
I20260620 18:46:58.957772 29415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756961096a5436c9478a423a41d4b8f" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34567 } }
I20260620 18:46:58.957835 29415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:46:58.957861 29415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:46:58.957938 29415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:46:58.958362 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 29347
I20260620 18:46:58.958487 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:46:58.958729 15669 external_mini_cluster.cc:1428] Setting key f5ca94f902a95f19f3d6ec3cc5c7d9de
I20260620 18:46:58.958719 29415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756961096a5436c9478a423a41d4b8f" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34567 } }
I20260620 18:46:58.958820 29415 leader_election.cc:302] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1756961096a5436c9478a423a41d4b8f; no voters:
I20260620 18:46:58.959008 29415 leader_election.cc:288] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:46:58.959055 29418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:46:58.959233 29418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [term 1 LEADER]: Becoming Leader. State: Replica: 1756961096a5436c9478a423a41d4b8f, State: Running, Role: LEADER
I20260620 18:46:58.959368 29415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:46:58.959383 29418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756961096a5436c9478a423a41d4b8f" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34567 } }
I20260620 18:46:58.959841 29418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1756961096a5436c9478a423a41d4b8f. Latest consensus state: current_term: 1 leader_uuid: "1756961096a5436c9478a423a41d4b8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756961096a5436c9478a423a41d4b8f" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34567 } } }
I20260620 18:46:58.959976 29418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:58.960005 29421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1756961096a5436c9478a423a41d4b8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1756961096a5436c9478a423a41d4b8f" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34567 } } }
I20260620 18:46:58.960136 29421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f [sys.catalog]: This master's current role is: LEADER
I20260620 18:46:58.960261 29429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:46:58.960812 29429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:46:58.962433 29429 catalog_manager.cc:1347] Generated new cluster ID: 16d735240b34416e824f9d7c45890280
I20260620 18:46:58.962481 29429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:46:58 dist-test-slave-tvl9 krb5kdc[29334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:46:58.969691 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:46:58.959394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.15.77.126:34567)
0620 18:46:58.959590 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 18:46:58.959602 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 18:46:58.959794 (+ 192us) server_negotiation.cc:407] Connection header received
0620 18:46:58.959981 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:46:58.959990 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:46:58.960060 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:46:58.960233 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:46:58.961341 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:58.962633 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:58.963453 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:46:58.963874 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:46:58.964015 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:46:58.966323 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:46:58.966327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:46:58.966327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:46:58.966340 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:46:58.968050 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:58.968440 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:58.968444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:58.968444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:58.968545 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:46:58.968762 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:46:58.968763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:46:58.968764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:46:58.969008 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:46:58.969181 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 18:46:58.969277 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 18:46:58.969523 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
I20260620 18:46:58.970788 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:46:58.996532 29429 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:46:58.997107 29429 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:46:59.002252 29429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1756961096a5436c9478a423a41d4b8f: Generated new TSK 0
I20260620 18:46:59.002475 29429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:46:59.024631 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 29347
2026-06-20T18:46:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1359 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:46:59 dist-test-slave-tvl9 krb5kdc[29442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:46:59 dist-test-slave-tvl9 krb5kdc[29442](info): set up 2 sockets
Jun 20 18:46:59 dist-test-slave-tvl9 krb5kdc[29442](info): commencing operation
krb5kdc: starting...
W20260620 18:47:00.063421 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:47:00 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:47:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:47:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:47:00.180109 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40491
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:00.262190 29455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:00.262457 29455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:00.262497 29455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:00.263789 29455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:47:00.263865 29455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:00.263886 29455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:00.263906 29455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:47:00.263933 29455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:47:00.265558 29455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:00.265870 29455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:00.266115 29455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:00.271399 29461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:00.271358 29463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:00.271530 29455 server_base.cc:1034] running on GCE node
W20260620 18:47:00.271358 29460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:00.271893 29455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:00.272213 29455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:00.273386 29455 hybrid_clock.cc:648] HybridClock initialized: now 1781981220273375 us; error 47 us; skew 500 ppm
Jun 20 18:47:00 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:00.275970 29455 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:47:00.276363 29455 webserver.cc:460] Webserver started at http://127.15.77.126:37085/ using document root <none> and password file <none>
I20260620 18:47:00.276611 29455 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:00.276657 29455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:00.276788 29455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:47:00.277853 29455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "2fd617cf38b84ad58404b3602b86a606"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "044db5f9d1897dcb8e1c3bc4d88a2f9f"
server_key_iv: "e6cff40616b550c26e9bcd0b68181aab"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.278213 29455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "2fd617cf38b84ad58404b3602b86a606"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "044db5f9d1897dcb8e1c3bc4d88a2f9f"
server_key_iv: "e6cff40616b550c26e9bcd0b68181aab"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.279713 29455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:47:00.280543 29470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.280771 29455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:47:00.280882 29455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "2fd617cf38b84ad58404b3602b86a606"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "044db5f9d1897dcb8e1c3bc4d88a2f9f"
server_key_iv: "e6cff40616b550c26e9bcd0b68181aab"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.280965 29455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:00.292958 29455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:00.294976 29455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:00.295138 29455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:00.299242 29455 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:40491
I20260620 18:47:00.299297 29522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:40491 every 8 connection(s)
I20260620 18:47:00.299736 29455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:47:00.300732 29523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:00.303031 29523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Bootstrap starting.
I20260620 18:47:00.303687 29523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:00.303923 29523 log.cc:824] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:00.304625 29523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: No bootstrap required, opened a new log
I20260620 18:47:00.304838 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 29455
I20260620 18:47:00.304942 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:47:00.305136 15669 external_mini_cluster.cc:1428] Setting key 2e679fd3fba357e1a43611eef2a005b5
I20260620 18:47:00.306708 29523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:00.306876 29523 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:00.306941 29523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd617cf38b84ad58404b3602b86a606, State: Initialized, Role: FOLLOWER
I20260620 18:47:00.307116 29523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:00.307200 29523 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:47:00.307224 29523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:47:00.307277 29523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:00.308391 29523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:00.308517 29523 leader_election.cc:302] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fd617cf38b84ad58404b3602b86a606; no voters:
I20260620 18:47:00.308720 29523 leader_election.cc:288] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:47:00.308750 29528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:47:00.308863 29528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd617cf38b84ad58404b3602b86a606, State: Running, Role: LEADER
I20260620 18:47:00.309006 29523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:47:00.309006 29528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:00.309502 29530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fd617cf38b84ad58404b3602b86a606. Latest consensus state: current_term: 1 leader_uuid: "2fd617cf38b84ad58404b3602b86a606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } } }
I20260620 18:47:00.309597 29530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: This master's current role is: LEADER
I20260620 18:47:00.309829 29529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fd617cf38b84ad58404b3602b86a606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } } }
I20260620 18:47:00.310171 29529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: This master's current role is: LEADER
I20260620 18:47:00.310369 29535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:47:00.311012 29535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:47:00.312835 29535 catalog_manager.cc:1347] Generated new cluster ID: a07d44627b194021b58e364eb7755f11
I20260620 18:47:00.312893 29535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:47:00 dist-test-slave-tvl9 krb5kdc[29442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:00.317809 29527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.305668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40722 (local address 127.15.77.126:40491)
0620 18:47:00.305853 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:00.305861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:00.305886 (+ 25us) server_negotiation.cc:407] Connection header received
0620 18:47:00.306699 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:00.306707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:00.306764 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:00.306942 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:00.307751 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.308829 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.309863 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.310515 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.310622 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.313942 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:00.313946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:00.313947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:00.313960 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:00.316089 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:00.316469 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:00.316473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:00.316473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:00.316604 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:00.316784 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:00.316785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:00.316785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:00.317081 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:00.317205 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:00.317443 (+ 238us) server_negotiation.cc:299] Negotiation successful
0620 18:47:00.317596 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260620 18:47:00.324148 29535 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:47:00.324931 29535 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:47:00.336021 29535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Generated new TSK 0
I20260620 18:47:00.336254 29535 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:47:00.371610 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40491
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:00.456784 29549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:00.457019 29549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:00.457044 29549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:00.458268 29549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:00.458312 29549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:00.458339 29549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:47:00.459822 29549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:00.460053 29549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:00.460240 29549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:00.465322 29555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:00.465447 29549 server_base.cc:1034] running on GCE node
W20260620 18:47:00.465341 29557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:00.465322 29554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:00.465790 29549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:00.466060 29549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:00.467232 29549 hybrid_clock.cc:648] HybridClock initialized: now 1781981220467228 us; error 42 us; skew 500 ppm
Jun 20 18:47:00 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:00.469858 29549 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:47:00.470315 29549 webserver.cc:460] Webserver started at http://127.15.77.65:41031/ using document root <none> and password file <none>
I20260620 18:47:00.470566 29549 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:00.470635 29549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:00.470789 29549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:47:00.471961 29549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "8ef88e1f9d674f4880390b6d8280e8a4"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "77b4f7eb66a1b9f0860af77bdfa46551"
server_key_iv: "a1cfaf63e514384d6fa901bb16961884"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.472332 29549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "8ef88e1f9d674f4880390b6d8280e8a4"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "77b4f7eb66a1b9f0860af77bdfa46551"
server_key_iv: "a1cfaf63e514384d6fa901bb16961884"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.473742 29549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:47:00.474483 29564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.474656 29549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:47:00.474766 29549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "8ef88e1f9d674f4880390b6d8280e8a4"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "77b4f7eb66a1b9f0860af77bdfa46551"
server_key_iv: "a1cfaf63e514384d6fa901bb16961884"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.474846 29549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:00.499624 29549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:00.501762 29549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:00.501937 29549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:00.502187 29549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:00.502570 29549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:00.502635 29549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.502681 29549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:00.502709 29549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.508797 29549 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:34055
I20260620 18:47:00.508944 29677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:34055 every 8 connection(s)
I20260620 18:47:00.509238 29549 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
Jun 20 18:47:00 dist-test-slave-tvl9 krb5kdc[29442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:00.518180 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 29549
I20260620 18:47:00.518314 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:47:00.518585 15669 external_mini_cluster.cc:1428] Setting key 5d9eddc14c8b93daac20dd51f58e4f7b
I20260620 18:47:00.519487 29527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.510009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:47957 (local address 127.15.77.126:40491)
0620 18:47:00.510055 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:00.510058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:00.510266 (+ 208us) server_negotiation.cc:407] Connection header received
0620 18:47:00.510964 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:00.510965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:00.510981 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:00.511044 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:00.512050 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.512622 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.513655 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.513897 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.513998 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.516267 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:00.516269 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:00.516269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:00.516276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:00.517772 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:00.518230 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:00.518231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:00.518232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:00.518293 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:00.518680 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:00.518681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:00.518682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:00.518855 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:00.518939 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:00.519264 (+ 325us) server_negotiation.cc:299] Negotiation successful
0620 18:47:00.519387 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:47:00.519773 29680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.509960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:47957)
0620 18:47:00.510135 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:00.510173 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:00.510835 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:00.511117 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:00.511121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:00.511740 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:00.511941 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.511946 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.512734 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.512735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.513554 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.513556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.514012 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.514012 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.514145 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.514730 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:00.514739 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:00.516157 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:00.517879 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:00.517887 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:00.517890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:00.518140 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:00.518389 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:00.518390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:00.518391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:00.518589 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:00.518932 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:00.518940 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:00.519090 (+ 150us) client_negotiation.cc:769] Sending connection context
0620 18:47:00.519219 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 18:47:00.519466 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260620 18:47:00.520370 29678 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40491
I20260620 18:47:00.520476 29678 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:00.520679 29678 heartbeater.cc:511] Master 127.15.77.126:40491 requested a full tablet report, sending...
I20260620 18:47:00.521323 29487 ts_manager.cc:194] Registered new tserver with Master: 8ef88e1f9d674f4880390b6d8280e8a4 (127.15.77.65:34055)
I20260620 18:47:00.522276 29487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:47957
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:47:00.545850 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40491
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:00.627419 29683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:00.627626 29683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:00.627656 29683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:00.628824 29683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:00.628865 29683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:00.628890 29683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:47:00.630368 29683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:00.630616 29683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:00.630851 29683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:00.635798 29689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:00.635828 29683 server_base.cc:1034] running on GCE node
W20260620 18:47:00.635869 29688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:00.635798 29691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:00.636180 29683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:00.636425 29683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:00.637586 29683 hybrid_clock.cc:648] HybridClock initialized: now 1781981220637574 us; error 35 us; skew 500 ppm
Jun 20 18:47:00 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:00.640127 29683 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:47:00.640570 29683 webserver.cc:460] Webserver started at http://127.15.77.66:37303/ using document root <none> and password file <none>
I20260620 18:47:00.640805 29683 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:00.640872 29683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:00.641031 29683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:47:00.642110 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "2a22635c494545e39a51e923fc1aba7d"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "bf1d3b50e376bde440175ce6c983f47a"
server_key_iv: "85155723fa320a4a5b182bd0e7af9a39"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.642475 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "2a22635c494545e39a51e923fc1aba7d"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "bf1d3b50e376bde440175ce6c983f47a"
server_key_iv: "85155723fa320a4a5b182bd0e7af9a39"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.643924 29683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:47:00.644690 29698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.644869 29683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.644974 29683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "2a22635c494545e39a51e923fc1aba7d"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "bf1d3b50e376bde440175ce6c983f47a"
server_key_iv: "85155723fa320a4a5b182bd0e7af9a39"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.645061 29683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:00.653551 29683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:00.656490 29683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:00.656646 29683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:00.656906 29683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:00.657263 29683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:00.657307 29683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.657337 29683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:00.657364 29683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.663653 29683 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:39677
I20260620 18:47:00.663710 29811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:39677 every 8 connection(s)
I20260620 18:47:00.664037 29683 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
Jun 20 18:47:00 dist-test-slave-tvl9 krb5kdc[29442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:00.670461 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 29683
I20260620 18:47:00.670574 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:47:00.670783 15669 external_mini_cluster.cc:1428] Setting key 9537117ac95c97ce6a3d76cce3a9de50
I20260620 18:47:00.673651 29527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.664756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:52043 (local address 127.15.77.126:40491)
0620 18:47:00.664824 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:00.664826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:00.664994 (+ 168us) server_negotiation.cc:407] Connection header received
0620 18:47:00.665496 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:00.665497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:00.665507 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:00.665570 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:00.666460 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.666965 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.668105 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.668301 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.668370 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.670583 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:00.670586 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:00.670586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:00.670595 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:00.672212 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:00.672644 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:00.672645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:00.672646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:00.672710 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:00.672978 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:00.672979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:00.672979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:00.673105 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:00.673179 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:00.673464 (+ 285us) server_negotiation.cc:299] Negotiation successful
0620 18:47:00.673555 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 18:47:00.673866 29814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.664718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:52043)
0620 18:47:00.664876 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:00.664918 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:00.665412 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:00.665626 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:00.665629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:00.666188 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:00.666382 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.666386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.667053 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.667053 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.667997 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.668000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.668411 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.668413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.668536 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.669049 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:00.669058 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:00.670459 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:00.672318 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:00.672326 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:00.672328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:00.672562 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:00.672789 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:00.672790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:00.672790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:00.672902 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:00.673181 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:00.673189 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:00.673332 (+ 143us) client_negotiation.cc:769] Sending connection context
0620 18:47:00.673442 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 18:47:00.673616 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260620 18:47:00.674446 29812 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40491
I20260620 18:47:00.674554 29812 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:00.674763 29812 heartbeater.cc:511] Master 127.15.77.126:40491 requested a full tablet report, sending...
I20260620 18:47:00.675140 29487 ts_manager.cc:194] Registered new tserver with Master: 2a22635c494545e39a51e923fc1aba7d (127.15.77.66:39677)
I20260620 18:47:00.675580 29487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:52043
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:47:00.698303 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40491
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:00.780949 29817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:00.781173 29817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:00.781199 29817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:00.782331 29817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:00.782367 29817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:00.782392 29817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:47:00.783994 29817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:40491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:00.784236 29817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:00.784451 29817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:00.789948 29825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:00.790012 29822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:00.790189 29817 server_base.cc:1034] running on GCE node
W20260620 18:47:00.789970 29823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:00.790479 29817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:00.790828 29817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:00.792008 29817 hybrid_clock.cc:648] HybridClock initialized: now 1781981220791990 us; error 43 us; skew 500 ppm
Jun 20 18:47:00 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:00.794521 29817 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:47:00.794957 29817 webserver.cc:460] Webserver started at http://127.15.77.67:35589/ using document root <none> and password file <none>
I20260620 18:47:00.795190 29817 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:00.795258 29817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:00.795436 29817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:47:00.796584 29817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "0af25ab364a746b0bf667ebb78ccf4e5"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "fa927f4ec289fcfad53be9de6ccbfe21"
server_key_iv: "579efc1ad7973a193bbe8b7d6c57587b"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.796941 29817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "0af25ab364a746b0bf667ebb78ccf4e5"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "fa927f4ec289fcfad53be9de6ccbfe21"
server_key_iv: "579efc1ad7973a193bbe8b7d6c57587b"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.798265 29817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:47:00.799038 29832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.799225 29817 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.799373 29817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "0af25ab364a746b0bf667ebb78ccf4e5"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "fa927f4ec289fcfad53be9de6ccbfe21"
server_key_iv: "579efc1ad7973a193bbe8b7d6c57587b"
server_key_version: "encryptionkey@0"
I20260620 18:47:00.799475 29817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:00.816915 29817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:00.818799 29817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:00.818980 29817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:00.819227 29817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:00.819610 29817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:00.819667 29817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.819713 29817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:00.819737 29817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:00.825505 29817 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:34417
I20260620 18:47:00.825580 29945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:34417 every 8 connection(s)
I20260620 18:47:00.825920 29817 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
Jun 20 18:47:00 dist-test-slave-tvl9 krb5kdc[29442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:00.833621 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 29817
I20260620 18:47:00.833724 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:47:00.833931 15669 external_mini_cluster.cc:1428] Setting key d0b85564e8a3d6d0ff11c3f446e1d40b
I20260620 18:47:00.835407 29527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.826683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:38863 (local address 127.15.77.126:40491)
0620 18:47:00.826749 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:00.826751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:00.827000 (+ 249us) server_negotiation.cc:407] Connection header received
0620 18:47:00.827502 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:00.827503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:00.827514 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:00.827560 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:00.828472 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.828958 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.829936 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.830116 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.830172 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.832233 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:00.832234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:00.832235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:00.832243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:00.833860 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:00.834377 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:00.834378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:00.834378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:00.834458 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:00.834739 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:00.834739 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:00.834740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:00.834908 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:00.834972 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:00.835223 (+ 251us) server_negotiation.cc:299] Negotiation successful
0620 18:47:00.835286 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 18:47:00.835659 29948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.826682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:38863)
0620 18:47:00.826871 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:00.826910 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:00.827415 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:00.827624 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:00.827627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:00.828189 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:00.828389 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.828394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.829046 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.829047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.829858 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.829860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.830194 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.830195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.830293 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.830760 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:00.830768 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:00.832125 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:00.833937 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:00.833944 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:00.833945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:00.834283 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:00.834533 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:00.834534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:00.834534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:00.834670 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:00.834982 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:00.834987 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:00.835106 (+ 119us) client_negotiation.cc:769] Sending connection context
0620 18:47:00.835193 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 18:47:00.835328 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260620 18:47:00.836179 29946 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40491
I20260620 18:47:00.836279 29946 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:00.836475 29946 heartbeater.cc:511] Master 127.15.77.126:40491 requested a full tablet report, sending...
I20260620 18:47:00.836833 29487 ts_manager.cc:194] Registered new tserver with Master: 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417)
I20260620 18:47:00.837211 29487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:38863
I20260620 18:47:00.845780 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:47:00.853320 29527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.847234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40734 (local address 127.15.77.126:40491)
0620 18:47:00.847303 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:00.847306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:00.847438 (+ 132us) server_negotiation.cc:407] Connection header received
0620 18:47:00.847517 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:00.847518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:00.847528 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:00.847577 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:00.848158 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.848705 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.849439 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.849640 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.849712 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.850936 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:00.850937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:00.850938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:00.850945 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:00.852161 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:00.852498 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:00.852499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:00.852500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:00.852560 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:00.852760 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:00.852761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:00.852761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:00.852915 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:00.853020 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:00.853129 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 18:47:00.853209 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 18:47:00.854842 29487 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:47:00.856520 29487 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:47:00.863170 29962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.859266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34008 (local address 127.15.77.65:34055)
0620 18:47:00.859580 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:00.859585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:00.859595 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:47:00.859633 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:00.859635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:00.859649 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:00.859726 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:00.859921 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.860696 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.862109 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.862754 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.862842 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.862902 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:00.863044 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 18:47:00.863086 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":122,"threads_started":1}
I20260620 18:47:00.863174 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.858997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34055 (local address 127.0.0.1:34008)
0620 18:47:00.859547 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:00.859554 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:00.859579 (+ 25us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:00.859740 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:00.859744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:00.859746 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:00.859832 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.859835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.860805 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.860806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.862014 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.862017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.862841 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.862842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.862920 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.862934 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 18:47:00.863028 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 18:47:00.863087 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260620 18:47:00.863276 29958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.858997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:39677 (local address 127.0.0.1:34258)
0620 18:47:00.859401 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:00.859422 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:00.859475 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:00.860017 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:00.860018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:00.860020 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:00.860054 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.860055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.861223 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.861224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.862109 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.862112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.863027 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.863028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.863095 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.863100 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:47:00.863151 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 18:47:00.863193 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":93,"threads_started":1}
I20260620 18:47:00.863549 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.859242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.15.77.66:39677)
0620 18:47:00.859857 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:00.859860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:00.859869 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:47:00.859908 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:00.859910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:00.859926 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:00.860006 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:00.860116 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.861143 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.862195 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.862871 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.862939 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.862990 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:00.863411 (+ 421us) server_negotiation.cc:299] Negotiation successful
0620 18:47:00.863462 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":134,"threads_started":1}
I20260620 18:47:00.863579 29957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.858993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:34417 (local address 127.0.0.1:41600)
0620 18:47:00.859149 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:00.859192 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:00.859447 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:00.859807 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:00.859808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:00.859810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:00.859867 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.859869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.861190 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.861191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.862372 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.862374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.863350 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.863351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.863412 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.863416 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:47:00.863475 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 18:47:00.863509 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":3,"thread_start_us":56,"threads_started":1}
I20260620 18:47:00.863624 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.859102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.15.77.67:34417)
0620 18:47:00.859286 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:00.859291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:00.859544 (+ 253us) server_negotiation.cc:407] Connection header received
0620 18:47:00.859598 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:00.859601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:00.859639 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:00.859716 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:00.859952 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.861075 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.862466 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.863185 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.863243 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.863292 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:00.863507 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 18:47:00.863550 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
I20260620 18:47:00.864344 29880 tablet_service.cc:1467] Processing CreateTablet for tablet 6957a19195a9486397827762f4dfa03a (DEFAULT_TABLE table=test-table [id=a62fcf6a6b6c42e38cda45146ecb9c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:00.864480 29612 tablet_service.cc:1467] Processing CreateTablet for tablet 6957a19195a9486397827762f4dfa03a (DEFAULT_TABLE table=test-table [id=a62fcf6a6b6c42e38cda45146ecb9c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:00.864480 29746 tablet_service.cc:1467] Processing CreateTablet for tablet 6957a19195a9486397827762f4dfa03a (DEFAULT_TABLE table=test-table [id=a62fcf6a6b6c42e38cda45146ecb9c65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:00.864748 29880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6957a19195a9486397827762f4dfa03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:00.864816 29612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6957a19195a9486397827762f4dfa03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:00.864815 29746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6957a19195a9486397827762f4dfa03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:00.867451 29963 tablet_bootstrap.cc:492] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: Bootstrap starting.
I20260620 18:47:00.868036 29965 tablet_bootstrap.cc:492] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: Bootstrap starting.
I20260620 18:47:00.868203 29963 tablet_bootstrap.cc:654] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:00.868757 29965 tablet_bootstrap.cc:654] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:00.868754 29963 log.cc:824] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:00.869320 29965 log.cc:824] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:00.869544 29963 tablet_bootstrap.cc:492] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: No bootstrap required, opened a new log
I20260620 18:47:00.869618 29963 ts_tablet_manager.cc:1397] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:47:00.869657 29964 tablet_bootstrap.cc:492] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Bootstrap starting.
I20260620 18:47:00.870287 29964 tablet_bootstrap.cc:654] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:00.870661 29965 tablet_bootstrap.cc:492] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: No bootstrap required, opened a new log
I20260620 18:47:00.870733 29965 ts_tablet_manager.cc:1397] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 18:47:00.871801 29964 log.cc:824] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:00.871759 29963 raft_consensus.cc:348] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:00.871959 29963 raft_consensus.cc:374] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:00.872020 29963 raft_consensus.cc:729] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af25ab364a746b0bf667ebb78ccf4e5, State: Initialized, Role: FOLLOWER
I20260620 18:47:00.872151 29963 consensus_queue.cc:260] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:00.872428 29963 ts_tablet_manager.cc:1428] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 18:47:00.872440 29946 heartbeater.cc:503] Master 127.15.77.126:40491 was elected leader, sending a full tablet report...
I20260620 18:47:00.872733 29965 raft_consensus.cc:348] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:00.872910 29965 raft_consensus.cc:374] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:00.872953 29965 raft_consensus.cc:729] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a22635c494545e39a51e923fc1aba7d, State: Initialized, Role: FOLLOWER
I20260620 18:47:00.873062 29965 consensus_queue.cc:260] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:00.873349 29965 ts_tablet_manager.cc:1428] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:47:00.873360 29812 heartbeater.cc:503] Master 127.15.77.126:40491 was elected leader, sending a full tablet report...
I20260620 18:47:00.873989 29964 tablet_bootstrap.cc:492] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: No bootstrap required, opened a new log
I20260620 18:47:00.874070 29964 ts_tablet_manager.cc:1397] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 18:47:00.875671 29964 raft_consensus.cc:348] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:00.875809 29964 raft_consensus.cc:374] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:00.875852 29964 raft_consensus.cc:729] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef88e1f9d674f4880390b6d8280e8a4, State: Initialized, Role: FOLLOWER
I20260620 18:47:00.875958 29964 consensus_queue.cc:260] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:00.876209 29964 ts_tablet_manager.cc:1428] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:47:00.876233 29678 heartbeater.cc:503] Master 127.15.77.126:40491 was elected leader, sending a full tablet report...
W20260620 18:47:00.914635 29813 tablet.cc:2369] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:47:00.993734 29971 raft_consensus.cc:482] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:47:00.993872 29971 raft_consensus.cc:504] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:00.994230 29971 leader_election.cc:288] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417), 2a22635c494545e39a51e923fc1aba7d (127.15.77.66:39677)
I20260620 18:47:00.997157 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.994565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:52697 (local address 127.15.77.66:39677)
0620 18:47:00.994617 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:00.994619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:00.994629 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:47:00.994661 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:00.994662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:00.994671 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:00.994748 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:00.994919 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.995436 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.996392 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.996783 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.996870 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.996880 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:00.997050 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 18:47:00.997089 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 18:47:00.997157 29680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.994422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:39677 (local address 127.15.77.65:52697)
0620 18:47:00.994499 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:00.994512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:00.994583 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:00.994754 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:00.994755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:00.994757 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:00.994834 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.994836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.995556 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.995558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.996294 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.996296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.996880 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.996881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.996961 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.996965 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:47:00.997040 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 18:47:00.997084 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260620 18:47:00.997270 29972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.994422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:34417 (local address 127.15.77.65:40971)
0620 18:47:00.994605 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:00.994616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:00.994699 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:00.994859 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:00.994860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:00.994861 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:00.994943 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.994944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.995696 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.995698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.996426 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:00.996428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.996992 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.996993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:00.997066 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.997077 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 18:47:00.997146 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 18:47:00.997184 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":63,"thread_start_us":63,"threads_started":1}
I20260620 18:47:00.997257 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:00.994589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:40971 (local address 127.15.77.67:34417)
0620 18:47:00.994636 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:00.994638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:00.994683 (+ 45us) server_negotiation.cc:407] Connection header received
0620 18:47:00.994781 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:00.994782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:00.994789 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:00.994854 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:00.995012 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.995606 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.996522 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:00.996893 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:00.996988 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:00.996996 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:00.997154 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 18:47:00.997191 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 18:47:00.997537 29766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6957a19195a9486397827762f4dfa03a" candidate_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a22635c494545e39a51e923fc1aba7d" is_pre_election: true
I20260620 18:47:00.997571 29900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6957a19195a9486397827762f4dfa03a" candidate_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" is_pre_election: true
I20260620 18:47:00.997709 29766 raft_consensus.cc:2454] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef88e1f9d674f4880390b6d8280e8a4 in term 0.
I20260620 18:47:00.997738 29900 raft_consensus.cc:2454] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef88e1f9d674f4880390b6d8280e8a4 in term 0.
I20260620 18:47:00.997933 29567 leader_election.cc:302] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a22635c494545e39a51e923fc1aba7d, 8ef88e1f9d674f4880390b6d8280e8a4; no voters:
I20260620 18:47:00.998080 29971 raft_consensus.cc:2781] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:47:00.998137 29971 raft_consensus.cc:482] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:47:00.998162 29971 raft_consensus.cc:3037] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:00.998871 29971 raft_consensus.cc:504] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:00.999024 29971 leader_election.cc:288] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 1 election: Requested vote from peers 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417), 2a22635c494545e39a51e923fc1aba7d (127.15.77.66:39677)
I20260620 18:47:00.999236 29900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6957a19195a9486397827762f4dfa03a" candidate_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0af25ab364a746b0bf667ebb78ccf4e5"
I20260620 18:47:00.999238 29766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6957a19195a9486397827762f4dfa03a" candidate_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a22635c494545e39a51e923fc1aba7d"
I20260620 18:47:00.999351 29766 raft_consensus.cc:3037] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:00.999351 29900 raft_consensus.cc:3037] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:01.000278 29766 raft_consensus.cc:2454] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef88e1f9d674f4880390b6d8280e8a4 in term 1.
I20260620 18:47:01.000277 29900 raft_consensus.cc:2454] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef88e1f9d674f4880390b6d8280e8a4 in term 1.
I20260620 18:47:01.000449 29567 leader_election.cc:302] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a22635c494545e39a51e923fc1aba7d, 8ef88e1f9d674f4880390b6d8280e8a4; no voters:
I20260620 18:47:01.000560 29971 raft_consensus.cc:2781] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:47:01.000727 29971 raft_consensus.cc:686] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef88e1f9d674f4880390b6d8280e8a4, State: Running, Role: LEADER
I20260620 18:47:01.000829 29971 consensus_queue.cc:237] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:01.001744 29487 catalog_manager.cc:5495] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef88e1f9d674f4880390b6d8280e8a4 (127.15.77.65). New cstate: current_term: 1 leader_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } health_report { overall_health: UNKNOWN } } }
I20260620 18:47:01.007095 29962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.004748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.15.77.65:34055)
0620 18:47:01.004800 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.004803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.004812 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:47:01.004844 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.004845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.004863 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.004946 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:47:01.005063 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.005530 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.006292 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.006505 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.006580 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.006751 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:47:01.006858 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:47:01.006952 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.006977 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.006996 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 18:47:01.008948 29766 raft_consensus.cc:1261] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Refusing update from remote peer 8ef88e1f9d674f4880390b6d8280e8a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:47:01.008941 29900 raft_consensus.cc:1261] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5 [term 1 FOLLOWER]: Refusing update from remote peer 8ef88e1f9d674f4880390b6d8280e8a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:47:01.009331 29973 consensus_queue.cc:1035] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:47:01.009477 29971 consensus_queue.cc:1035] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 18:47:01.009895 29679 tablet.cc:2369] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:47:01.012156 29976 mvcc.cc:205] Tried to move back new op lower bound from 7298995081249816576 to 7298995081220235264. Current Snapshot: MvccSnapshot[applied={T|T < 7298995081249816576}]
I20260620 18:47:01.014968 29487 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:47:01.015077 29487 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:47:01.015825 29487 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Sending DeleteTablet for 3 replicas of tablet 6957a19195a9486397827762f4dfa03a
I20260620 18:47:01.016208 29612 tablet_service.cc:1514] Processing DeleteTablet for tablet 6957a19195a9486397827762f4dfa03a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:01 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:34008
I20260620 18:47:01.016225 29880 tablet_service.cc:1514] Processing DeleteTablet for tablet 6957a19195a9486397827762f4dfa03a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:01 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:41600
I20260620 18:47:01.016273 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 29549
I20260620 18:47:01.016794 29746 tablet_service.cc:1514] Processing DeleteTablet for tablet 6957a19195a9486397827762f4dfa03a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:01 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:34258
I20260620 18:47:01.017030 29987 tablet_replica.cc:330] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: stopping tablet replica
I20260620 18:47:01.017109 29987 raft_consensus.cc:2229] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:47:01.017179 29987 raft_consensus.cc:2258] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:01.017313 29986 tablet_replica.cc:330] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: stopping tablet replica
I20260620 18:47:01.017418 29986 raft_consensus.cc:2229] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:47:01.017520 29986 raft_consensus.cc:2258] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:01.017728 29987 ts_tablet_manager.cc:1905] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:47:01.017939 29986 ts_tablet_manager.cc:1905] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:47:01.019291 29987 ts_tablet_manager.cc:1918] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:47:01.019397 29987 log.cc:1196] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/6957a19195a9486397827762f4dfa03a
I20260620 18:47:01.019690 29987 ts_tablet_manager.cc:1939] T 6957a19195a9486397827762f4dfa03a P 2a22635c494545e39a51e923fc1aba7d: Deleting consensus metadata
I20260620 18:47:01.019799 29986 ts_tablet_manager.cc:1918] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:47:01.019872 29986 log.cc:1196] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/6957a19195a9486397827762f4dfa03a
I20260620 18:47:01.020143 29473 catalog_manager.cc:4841] TS 2a22635c494545e39a51e923fc1aba7d (127.15.77.66:39677): tablet 6957a19195a9486397827762f4dfa03a (table test-table [id=a62fcf6a6b6c42e38cda45146ecb9c65]) successfully deleted
I20260620 18:47:01.020195 29986 ts_tablet_manager.cc:1939] T 6957a19195a9486397827762f4dfa03a P 0af25ab364a746b0bf667ebb78ccf4e5: Deleting consensus metadata
I20260620 18:47:01.020567 29474 catalog_manager.cc:4841] TS 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417): tablet 6957a19195a9486397827762f4dfa03a (table test-table [id=a62fcf6a6b6c42e38cda45146ecb9c65]) successfully deleted
W20260620 18:47:01.023272 29472 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:34055 (error 108)
I20260620 18:47:01.023588 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 29683
I20260620 18:47:01.024148 29957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.023687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34055 (local address 127.0.0.1:34016)
0620 18:47:01.024065 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.024107 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357}
W20260620 18:47:01.025372 29472 catalog_manager.cc:4584] TS 8ef88e1f9d674f4880390b6d8280e8a4 (127.15.77.65:34055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6957a19195a9486397827762f4dfa03a: Network error: Client connection negotiation failed: client connection to 127.15.77.65:34055: connect: Connection refused (error 111)
I20260620 18:47:01.028770 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 29817
I20260620 18:47:01.033409 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 29455
I20260620 18:47:01.038748 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40491
--webserver_interface=127.15.77.126
--webserver_port=37085
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:01.120656 29989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:01.120864 29989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:01.120886 29989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:01.122080 29989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:47:01.122123 29989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:01.122136 29989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:01.122149 29989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:47:01.122161 29989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:47:01.123677 29989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=37085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:01.123930 29989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:01.124188 29989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:01.134788 29995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:01.134781 29996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:01.134864 29989 server_base.cc:1034] running on GCE node
W20260620 18:47:01.134786 29998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:01.135283 29989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:01.135732 29989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:01.136905 29989 hybrid_clock.cc:648] HybridClock initialized: now 1781981221136886 us; error 38 us; skew 500 ppm
Jun 20 18:47:01 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:01.146160 29989 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:47:01.146744 29989 webserver.cc:460] Webserver started at http://127.15.77.126:37085/ using document root <none> and password file <none>
I20260620 18:47:01.146996 29989 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:01.147058 29989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:01.148468 29989 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:01.149134 30005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.149314 29989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:47:01.149410 29989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "2fd617cf38b84ad58404b3602b86a606"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "044db5f9d1897dcb8e1c3bc4d88a2f9f"
server_key_iv: "e6cff40616b550c26e9bcd0b68181aab"
server_key_version: "encryptionkey@0"
I20260620 18:47:01.149791 29989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:01.158295 29989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:01.160540 29989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:01.160648 29989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:01.164489 29989 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:40491
I20260620 18:47:01.164561 30057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:40491 every 8 connection(s)
I20260620 18:47:01.164947 29989 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:47:01.166543 30058 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:47:01.167194 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Bootstrap starting.
I20260620 18:47:01.173666 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 29989
I20260620 18:47:01.173944 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:34055
--local_ip_for_outbound_sockets=127.15.77.65
--tserver_master_addrs=127.15.77.126:40491
--webserver_port=41031
--webserver_interface=127.15.77.65
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:47:01.175827 30058 log.cc:824] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:01.177851 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:47:01.178156 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Bootstrap complete.
I20260620 18:47:01.180546 30058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:01.180792 30058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd617cf38b84ad58404b3602b86a606, State: Initialized, Role: FOLLOWER
I20260620 18:47:01.180994 30058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:01.181090 30058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:47:01.181149 30058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:47:01.181198 30058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 1 FOLLOWER]: Advancing to term 2
I20260620 18:47:01.182201 30058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:01.182320 30058 leader_election.cc:302] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fd617cf38b84ad58404b3602b86a606; no voters:
I20260620 18:47:01.182570 30058 leader_election.cc:288] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 18:47:01.182819 30063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 18:47:01.182904 30058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:47:01.183141 30063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 2 LEADER]: Becoming Leader. State: Replica: 2fd617cf38b84ad58404b3602b86a606, State: Running, Role: LEADER
I20260620 18:47:01.183414 30063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:01.183801 30063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fd617cf38b84ad58404b3602b86a606. Latest consensus state: current_term: 2 leader_uuid: "2fd617cf38b84ad58404b3602b86a606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } } }
I20260620 18:47:01.183879 30064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2fd617cf38b84ad58404b3602b86a606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } } }
I20260620 18:47:01.183939 30063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: This master's current role is: LEADER
I20260620 18:47:01.183969 30064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: This master's current role is: LEADER
I20260620 18:47:01.184149 30071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:47:01.184945 30071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:47:01.185127 30071 catalog_manager.cc:1259] Loaded cluster ID: a07d44627b194021b58e364eb7755f11
I20260620 18:47:01.185328 30071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:47:01.185557 30071 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260620 18:47:01.190725 30071 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260620 18:47:01.193946 30071 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:47:01.194157 30071 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Loaded TSK: 0
I20260620 18:47:01.194371 30071 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 18:47:01.263450 30060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:01.263655 30060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:01.263679 30060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:01.264873 30060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:01.264916 30060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:01.264942 30060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:47:01.266405 30060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:34055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=41031
--tserver_master_addrs=127.15.77.126:40491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:01.266660 30060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:01.266919 30060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:01.273075 30088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:01.273070 30086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:01.273101 30085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:01.273401 30060 server_base.cc:1034] running on GCE node
I20260620 18:47:01.273610 30060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:01.273878 30060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:01.275034 30060 hybrid_clock.cc:648] HybridClock initialized: now 1781981221275010 us; error 41 us; skew 500 ppm
Jun 20 18:47:01 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:01.277796 30060 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:47:01.278219 30060 webserver.cc:460] Webserver started at http://127.15.77.65:41031/ using document root <none> and password file <none>
I20260620 18:47:01.278463 30060 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:01.278545 30060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:01.280059 30060 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:47:01.280786 30095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.280959 30060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:47:01.281071 30060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "8ef88e1f9d674f4880390b6d8280e8a4"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "77b4f7eb66a1b9f0860af77bdfa46551"
server_key_iv: "a1cfaf63e514384d6fa901bb16961884"
server_key_version: "encryptionkey@0"
I20260620 18:47:01.281410 30060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:01.307428 30060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:01.309913 30060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:01.310086 30060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:01.310395 30060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:01.310874 30102 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 18:47:01.312896 30060 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 18:47:01.312986 30060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 18:47:01.313030 30060 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 18:47:01.313820 30060 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 18:47:01.313872 30060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 18:47:01.313939 30102 tablet_bootstrap.cc:492] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Bootstrap starting.
I20260620 18:47:01.320936 30060 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:34055
I20260620 18:47:01.321048 30209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:34055 every 8 connection(s)
I20260620 18:47:01.321382 30060 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:47:01.325481 30102 log.cc:824] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:01.328183 30102 tablet_bootstrap.cc:492] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:47:01.328517 30102 tablet_bootstrap.cc:492] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Bootstrap complete.
Jun 20 18:47:01 dist-test-slave-tvl9 krb5kdc[29442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:01.329499 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 30060
I20260620 18:47:01.329537 30102 ts_tablet_manager.cc:1397] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260620 18:47:01.329782 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:39677
--local_ip_for_outbound_sockets=127.15.77.66
--tserver_master_addrs=127.15.77.126:40491
--webserver_port=37303
--webserver_interface=127.15.77.66
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:47:01.331213 30102 raft_consensus.cc:348] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:01.331629 30102 raft_consensus.cc:729] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef88e1f9d674f4880390b6d8280e8a4, State: Initialized, Role: FOLLOWER
I20260620 18:47:01.331842 30102 consensus_queue.cc:260] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } } peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } }
I20260620 18:47:01.332175 30102 ts_tablet_manager.cc:1428] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20260620 18:47:01.334069 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.322269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:43983 (local address 127.15.77.126:40491)
0620 18:47:01.322442 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.322458 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.322531 (+ 73us) server_negotiation.cc:407] Connection header received
0620 18:47:01.323562 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.323567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.323613 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.323720 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:01.324800 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.325767 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.326885 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.327530 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.327649 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.329993 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:01.329997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:01.329997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:01.330009 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:01.332120 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:01.332679 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:01.332681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:01.332681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:01.332784 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:01.333091 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:01.333091 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:01.333091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:01.333316 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:01.333382 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.333732 (+ 350us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.333845 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260620 18:47:01.334144 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.322177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:43983)
0620 18:47:01.322391 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.322439 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:01.323289 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:01.323832 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:01.323838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:01.324320 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:01.324642 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.324652 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.325882 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.325883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.326771 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.326774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.327642 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.327644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.327763 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.328375 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:01.328384 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:01.329873 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:01.332239 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:01.332247 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:01.332249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:01.332586 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:01.332866 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:01.332867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:01.332868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:01.333022 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:01.333388 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:01.333395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:01.333564 (+ 169us) client_negotiation.cc:769] Sending connection context
0620 18:47:01.333683 (+ 119us) client_negotiation.cc:240] Negotiation successful
0620 18:47:01.333876 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
I20260620 18:47:01.334919 30210 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40491
I20260620 18:47:01.335038 30210 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:01.335239 30210 heartbeater.cc:511] Master 127.15.77.126:40491 requested a full tablet report, sending...
I20260620 18:47:01.336122 30022 ts_manager.cc:194] Registered new tserver with Master: 8ef88e1f9d674f4880390b6d8280e8a4 (127.15.77.65:34055)
I20260620 18:47:01.337473 30022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:43983
I20260620 18:47:01.338383 30210 heartbeater.cc:503] Master 127.15.77.126:40491 was elected leader, sending a full tablet report...
I20260620 18:47:01.341341 30217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.336811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34055 (local address 127.0.0.1:34018)
0620 18:47:01.337810 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.337850 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:01.337923 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:01.338389 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:01.338393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:01.338396 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:01.338461 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.338464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.339528 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.339529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.340351 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.340354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.341011 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.341012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.341108 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.341118 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 18:47:01.341210 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 18:47:01.341271 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"thread_start_us":102,"threads_started":1}
I20260620 18:47:01.341341 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.336861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34018 (local address 127.15.77.65:34055)
0620 18:47:01.338086 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.338091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.338102 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:47:01.338137 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.338140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.338270 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.338360 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:01.338613 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.339444 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.340443 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.340908 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.341006 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.341062 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.341229 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.341271 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1189,"thread_start_us":70,"threads_started":1}
I20260620 18:47:01.341733 30144 tablet_service.cc:1514] Processing DeleteTablet for tablet 6957a19195a9486397827762f4dfa03a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:01 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:34018
I20260620 18:47:01.342229 30219 tablet_replica.cc:330] stopping tablet replica
I20260620 18:47:01.342342 30219 raft_consensus.cc:2229] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:47:01.342410 30219 raft_consensus.cc:2258] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:01.342710 30219 ts_tablet_manager.cc:1905] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:47:01.345068 30219 ts_tablet_manager.cc:1918] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:47:01.345146 30219 log.cc:1196] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/6957a19195a9486397827762f4dfa03a
I20260620 18:47:01.345374 30219 ts_tablet_manager.cc:1939] T 6957a19195a9486397827762f4dfa03a P 8ef88e1f9d674f4880390b6d8280e8a4: Deleting consensus metadata
I20260620 18:47:01.345889 30007 catalog_manager.cc:4841] TS 8ef88e1f9d674f4880390b6d8280e8a4 (127.15.77.65:34055): tablet 6957a19195a9486397827762f4dfa03a (table test-table [id=a62fcf6a6b6c42e38cda45146ecb9c65]) successfully deleted
W20260620 18:47:01.419680 30215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:01.419895 30215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:01.419919 30215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:01.421115 30215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:01.421159 30215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:01.421216 30215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:47:01.422696 30215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:39677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=37303
--tserver_master_addrs=127.15.77.126:40491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:01.422942 30215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:01.423180 30215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:01.428308 30224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:01.428314 30225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:01.428404 30227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:01.428357 30215 server_base.cc:1034] running on GCE node
I20260620 18:47:01.428728 30215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:01.429001 30215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:01.430161 30215 hybrid_clock.cc:648] HybridClock initialized: now 1781981221430147 us; error 32 us; skew 500 ppm
Jun 20 18:47:01 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:01.432813 30215 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:47:01.433326 30215 webserver.cc:460] Webserver started at http://127.15.77.66:37303/ using document root <none> and password file <none>
I20260620 18:47:01.433563 30215 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:01.433645 30215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:01.435001 30215 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:47:01.435671 30234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.435848 30215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.435949 30215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "2a22635c494545e39a51e923fc1aba7d"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "bf1d3b50e376bde440175ce6c983f47a"
server_key_iv: "85155723fa320a4a5b182bd0e7af9a39"
server_key_version: "encryptionkey@0"
I20260620 18:47:01.436319 30215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:01.447613 30215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:01.449884 30215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:01.450047 30215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:01.450273 30215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:01.450573 30215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:01.450624 30215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.450666 30215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:01.450707 30215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.456939 30215 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:39677
I20260620 18:47:01.456983 30347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:39677 every 8 connection(s)
I20260620 18:47:01.457327 30215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
Jun 20 18:47:01 dist-test-slave-tvl9 krb5kdc[29442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:01.466511 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.457970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:45439 (local address 127.15.77.126:40491)
0620 18:47:01.458020 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.458022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.458242 (+ 220us) server_negotiation.cc:407] Connection header received
0620 18:47:01.458783 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.458785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.458802 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.458864 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:01.459917 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.460426 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.461421 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.461640 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.461725 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.463876 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:01.463877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:01.463878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:01.463886 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:01.465117 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:01.465526 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:01.465527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:01.465528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:01.465588 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:01.465831 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:01.465832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:01.465833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:01.465952 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:01.466027 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.466319 (+ 292us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.466415 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:47:01.466696 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.457955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:45439)
0620 18:47:01.458091 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.458157 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:01.458673 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:01.458939 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:01.458943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:01.459613 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:01.459813 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.459817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.460531 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.460532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.461327 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.461329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.461739 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.461740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.461866 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.462365 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:01.462373 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:01.463775 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:01.465207 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:01.465214 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:01.465217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:01.465433 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:01.465653 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:01.465654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:01.465654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:01.465759 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:01.466019 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:01.466026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:01.466164 (+ 138us) client_negotiation.cc:769] Sending connection context
0620 18:47:01.466275 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 18:47:01.466445 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260620 18:47:01.466890 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 30215
I20260620 18:47:01.467159 30348 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40491
I20260620 18:47:01.467196 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:34417
--local_ip_for_outbound_sockets=127.15.77.67
--tserver_master_addrs=127.15.77.126:40491
--webserver_port=35589
--webserver_interface=127.15.77.67
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:47:01.467252 30348 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:01.467475 30348 heartbeater.cc:511] Master 127.15.77.126:40491 requested a full tablet report, sending...
I20260620 18:47:01.467870 30022 ts_manager.cc:194] Registered new tserver with Master: 2a22635c494545e39a51e923fc1aba7d (127.15.77.66:39677)
I20260620 18:47:01.468261 30022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:45439
W20260620 18:47:01.548161 30351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:01.548415 30351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:01.548454 30351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:01.549631 30351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:01.549706 30351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:01.549750 30351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:47:01.551232 30351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:34417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=35589
--tserver_master_addrs=127.15.77.126:40491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:01.551539 30351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:01.551811 30351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:01.556856 30357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:01.556885 30356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:01.556866 30359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:01.557164 30351 server_base.cc:1034] running on GCE node
I20260620 18:47:01.557349 30351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:01.557545 30351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:01.558688 30351 hybrid_clock.cc:648] HybridClock initialized: now 1781981221558677 us; error 27 us; skew 500 ppm
Jun 20 18:47:01 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:01.561195 30351 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:47:01.561622 30351 webserver.cc:460] Webserver started at http://127.15.77.67:35589/ using document root <none> and password file <none>
I20260620 18:47:01.561874 30351 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:01.561955 30351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:01.563350 30351 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:47:01.563969 30366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.564141 30351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:47:01.564246 30351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "0af25ab364a746b0bf667ebb78ccf4e5"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "fa927f4ec289fcfad53be9de6ccbfe21"
server_key_iv: "579efc1ad7973a193bbe8b7d6c57587b"
server_key_version: "encryptionkey@0"
I20260620 18:47:01.564599 30351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:01.586400 30351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:01.588433 30351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:01.588620 30351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:01.588886 30351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:01.589231 30351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:01.589290 30351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.589334 30351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:01.589358 30351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.595211 30351 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:34417
I20260620 18:47:01.595292 30479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:34417 every 8 connection(s)
I20260620 18:47:01.595695 30351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
Jun 20 18:47:01 dist-test-slave-tvl9 krb5kdc[29442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:01.602452 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 30351
I20260620 18:47:01.605636 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.596493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:55049 (local address 127.15.77.126:40491)
0620 18:47:01.596538 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.596540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.596767 (+ 227us) server_negotiation.cc:407] Connection header received
0620 18:47:01.597270 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.597272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.597284 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.597341 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:01.598269 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.598796 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.599979 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.600172 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.600227 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.602403 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:01.602405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:01.602406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:01.602415 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:01.604210 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:01.604680 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:01.604680 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:01.604681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:01.604734 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:01.605002 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:01.605003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:01.605003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:01.605132 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:01.605203 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.605446 (+ 243us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.605536 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 18:47:01.605762 30482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.596489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:55049)
0620 18:47:01.596626 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.596665 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:01.597160 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:01.597394 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:01.597397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:01.597961 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:01.598173 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.598179 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.598914 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.598915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.599895 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.599898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.600253 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.600254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.600365 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.600884 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:01.600893 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:01.602285 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:01.604300 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:01.604307 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:01.604310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:01.604601 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:01.604829 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:01.604830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:01.604830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:01.604936 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:01.605200 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:01.605204 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:01.605320 (+ 116us) client_negotiation.cc:769] Sending connection context
0620 18:47:01.605403 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 18:47:01.605528 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260620 18:47:01.606437 30480 heartbeater.cc:347] Connected to a master server at 127.15.77.126:40491
I20260620 18:47:01.606542 30480 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:01.606752 30480 heartbeater.cc:511] Master 127.15.77.126:40491 requested a full tablet report, sending...
I20260620 18:47:01.607201 30022 ts_manager.cc:194] Registered new tserver with Master: 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417)
I20260620 18:47:01.607591 30022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:55049
I20260620 18:47:01.610170 30484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.602915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43754 (local address 127.15.77.126:40491)
0620 18:47:01.603034 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.603036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.603185 (+ 149us) server_negotiation.cc:407] Connection header received
0620 18:47:01.603210 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.603211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.603222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.603274 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:01.604014 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.604712 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.605684 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.606057 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.606112 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.607537 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:01.607539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:01.607539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:01.607546 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:01.609010 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:01.609376 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:01.609377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:01.609378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:01.609434 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:01.609659 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:01.609659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:01.609660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:01.609769 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:01.609949 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.609987 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.610065 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260620 18:47:01.610962 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:47:01.613760 30484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.611382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43766 (local address 127.15.77.126:40491)
0620 18:47:01.611465 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.611467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.611563 (+ 96us) server_negotiation.cc:407] Connection header received
0620 18:47:01.611629 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.611629 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.611658 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.611723 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:47:01.611800 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.612260 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.613004 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.613210 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.613273 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.613405 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:47:01.613542 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:47:01.613616 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.613684 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.613706 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 18:47:01.614104 30022 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:43766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:47:01.616011 30022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:47:01.618444 30144 tablet_service.cc:1467] Processing CreateTablet for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f (DEFAULT_TABLE table=test-table [id=f35dfe2bca9049229f37a0437726d32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:01.618673 30144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:01.620349 30102 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Bootstrap starting.
I20260620 18:47:01.620700 30102 tablet_bootstrap.cc:654] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:01.621285 30102 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: No bootstrap required, opened a new log
I20260620 18:47:01.621335 30102 ts_tablet_manager.cc:1397] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:01.621440 30102 raft_consensus.cc:348] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.621505 30102 raft_consensus.cc:374] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:01.621547 30102 raft_consensus.cc:729] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef88e1f9d674f4880390b6d8280e8a4, State: Initialized, Role: FOLLOWER
I20260620 18:47:01.621589 30102 consensus_queue.cc:260] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.621666 30102 ts_tablet_manager.cc:1428] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.622390 30217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.618146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:34417 (local address 127.0.0.1:44020)
0620 18:47:01.618197 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.618210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:01.618252 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:01.618883 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:01.618884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:01.618885 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:01.618921 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.618923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.620012 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.620013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.621134 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.621137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.622126 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.622127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.622202 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.622206 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:47:01.622257 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 18:47:01.622311 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":5}
I20260620 18:47:01.622526 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.618184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:39677 (local address 127.0.0.1:35688)
0620 18:47:01.618336 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.618348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:01.618399 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:01.618855 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:01.618856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:01.618859 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:01.618921 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.618923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.620283 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.620284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.621321 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.621324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.622258 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.622259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.622329 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.622332 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:47:01.622386 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 18:47:01.622442 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":103,"threads_started":1}
I20260620 18:47:01.622572 30487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.618455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.15.77.66:39677)
0620 18:47:01.618682 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.618687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.618698 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:47:01.618742 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.618745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.618762 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.618845 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:01.619094 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.620167 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.621417 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.622088 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.622139 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.622193 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.622423 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.622474 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
I20260620 18:47:01.622792 30488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.618497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.15.77.67:34417)
0620 18:47:01.618707 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.618711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.618722 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:47:01.618763 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.618766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.618782 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.618994 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:01.619067 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.619904 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.621208 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.622003 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.622125 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.622193 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.622640 (+ 447us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.622694 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":102,"threads_started":1}
I20260620 18:47:01.623456 30282 tablet_service.cc:1467] Processing CreateTablet for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f (DEFAULT_TABLE table=test-table [id=f35dfe2bca9049229f37a0437726d32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:01.623481 30414 tablet_service.cc:1467] Processing CreateTablet for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f (DEFAULT_TABLE table=test-table [id=f35dfe2bca9049229f37a0437726d32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:01.623828 30282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:01.623862 30414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:01.626492 30490 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Bootstrap starting.
I20260620 18:47:01.626682 30491 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Bootstrap starting.
I20260620 18:47:01.627174 30490 tablet_bootstrap.cc:654] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:01.627207 30491 tablet_bootstrap.cc:654] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:01.627699 30491 log.cc:824] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:01.627713 30490 log.cc:824] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:01.628602 30491 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: No bootstrap required, opened a new log
I20260620 18:47:01.628602 30490 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: No bootstrap required, opened a new log
I20260620 18:47:01.628680 30490 ts_tablet_manager.cc:1397] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 18:47:01.628679 30491 ts_tablet_manager.cc:1397] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 18:47:01.630661 30491 raft_consensus.cc:348] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.630661 30490 raft_consensus.cc:348] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.630836 30490 raft_consensus.cc:374] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:01.630836 30491 raft_consensus.cc:374] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:01.630872 30490 raft_consensus.cc:729] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a22635c494545e39a51e923fc1aba7d, State: Initialized, Role: FOLLOWER
I20260620 18:47:01.630872 30491 raft_consensus.cc:729] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af25ab364a746b0bf667ebb78ccf4e5, State: Initialized, Role: FOLLOWER
I20260620 18:47:01.630998 30490 consensus_queue.cc:260] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.630998 30491 consensus_queue.cc:260] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.631294 30491 ts_tablet_manager.cc:1428] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 18:47:01.631294 30490 ts_tablet_manager.cc:1428] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 18:47:01.631345 30348 heartbeater.cc:503] Master 127.15.77.126:40491 was elected leader, sending a full tablet report...
I20260620 18:47:01.631652 30480 heartbeater.cc:503] Master 127.15.77.126:40491 was elected leader, sending a full tablet report...
I20260620 18:47:01.634055 30216 raft_consensus.cc:482] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:47:01.634186 30216 raft_consensus.cc:504] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.634543 30216 leader_election.cc:288] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417), 2a22635c494545e39a51e923fc1aba7d (127.15.77.66:39677)
I20260620 18:47:01.637353 30487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.634727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:36751 (local address 127.15.77.66:39677)
0620 18:47:01.634769 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.634771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.634782 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:47:01.634849 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.634850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.634860 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.634911 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:01.635106 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.635658 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.636623 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.636996 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.637090 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.637099 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.637249 (+ 150us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.637284 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:47:01.637372 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.634683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:39677 (local address 127.15.77.65:36751)
0620 18:47:01.634746 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.634758 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:01.634799 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:01.634956 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:01.634957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:01.634959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:01.635026 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.635028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.635754 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.635755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.636525 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.636527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.637090 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.637091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.637156 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.637159 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:47:01.637240 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 18:47:01.637282 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 18:47:01.637691 30488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.634875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:59325 (local address 127.15.77.67:34417)
0620 18:47:01.634918 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.634920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.634928 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:47:01.635004 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.635005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.635014 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.635069 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:01.635572 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.636023 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.636935 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.637303 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.637399 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.637407 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.637593 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.637632 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:47:01.637758 30302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bba3fb4fb6c4f4b8fab8da67b8d816f" candidate_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a22635c494545e39a51e923fc1aba7d" is_pre_election: true
I20260620 18:47:01.637916 30302 raft_consensus.cc:2454] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef88e1f9d674f4880390b6d8280e8a4 in term 0.
I20260620 18:47:01.637928 30496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.634684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:34417 (local address 127.15.77.65:59325)
0620 18:47:01.634864 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.634875 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:01.634926 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:01.635073 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:01.635073 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:01.635075 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:01.635492 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.635495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.636106 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.636107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.636833 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:01.636835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.637402 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.637403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:01.637480 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.637490 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 18:47:01.637798 (+ 308us) client_negotiation.cc:240] Negotiation successful
0620 18:47:01.637843 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":23,"thread_start_us":106,"threads_started":1}
I20260620 18:47:01.638181 30098 leader_election.cc:302] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a22635c494545e39a51e923fc1aba7d, 8ef88e1f9d674f4880390b6d8280e8a4; no voters:
I20260620 18:47:01.638254 30434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bba3fb4fb6c4f4b8fab8da67b8d816f" candidate_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" is_pre_election: true
I20260620 18:47:01.638311 30216 raft_consensus.cc:2781] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:47:01.638353 30216 raft_consensus.cc:482] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:47:01.638384 30216 raft_consensus.cc:3037] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:01.638402 30434 raft_consensus.cc:2454] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef88e1f9d674f4880390b6d8280e8a4 in term 0.
I20260620 18:47:01.639201 30216 raft_consensus.cc:504] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.639370 30216 leader_election.cc:288] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 1 election: Requested vote from peers 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417), 2a22635c494545e39a51e923fc1aba7d (127.15.77.66:39677)
I20260620 18:47:01.639567 30302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bba3fb4fb6c4f4b8fab8da67b8d816f" candidate_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a22635c494545e39a51e923fc1aba7d"
I20260620 18:47:01.639648 30302 raft_consensus.cc:3037] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:01.639544 30434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bba3fb4fb6c4f4b8fab8da67b8d816f" candidate_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0af25ab364a746b0bf667ebb78ccf4e5"
I20260620 18:47:01.639703 30434 raft_consensus.cc:3037] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:01.640405 30434 raft_consensus.cc:2454] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef88e1f9d674f4880390b6d8280e8a4 in term 1.
I20260620 18:47:01.640488 30302 raft_consensus.cc:2454] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef88e1f9d674f4880390b6d8280e8a4 in term 1.
I20260620 18:47:01.640635 30099 leader_election.cc:302] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0af25ab364a746b0bf667ebb78ccf4e5, 8ef88e1f9d674f4880390b6d8280e8a4; no voters:
I20260620 18:47:01.640805 30216 raft_consensus.cc:2781] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:47:01.640969 30216 raft_consensus.cc:686] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef88e1f9d674f4880390b6d8280e8a4, State: Running, Role: LEADER
I20260620 18:47:01.641050 30216 consensus_queue.cc:237] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.641825 30022 catalog_manager.cc:5495] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef88e1f9d674f4880390b6d8280e8a4 (127.15.77.65). New cstate: current_term: 1 leader_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } health_report { overall_health: HEALTHY } } }
I20260620 18:47:01.650460 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.648076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.15.77.65:34055)
0620 18:47:01.648117 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:01.648119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:01.648157 (+ 38us) server_negotiation.cc:407] Connection header received
0620 18:47:01.648248 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:01.648249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:01.648270 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:01.648348 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:47:01.648452 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.648916 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.649670 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:01.649859 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:01.649944 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:01.650121 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:47:01.650225 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:47:01.650309 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:01.650350 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 18:47:01.650375 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:47:01.652139 30434 raft_consensus.cc:1261] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 1 FOLLOWER]: Refusing update from remote peer 8ef88e1f9d674f4880390b6d8280e8a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:47:01.652154 30302 raft_consensus.cc:1261] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Refusing update from remote peer 8ef88e1f9d674f4880390b6d8280e8a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:47:01.652518 30216 consensus_queue.cc:1035] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:47:01.652666 30497 consensus_queue.cc:1035] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:47:01.655105 30501 mvcc.cc:205] Tried to move back new op lower bound from 7298995083884253184 to 7298995083842289664. Current Snapshot: MvccSnapshot[applied={T|T < 7298995083884253184}]
I20260620 18:47:01.655165 30500 mvcc.cc:205] Tried to move back new op lower bound from 7298995083884253184 to 7298995083842289664. Current Snapshot: MvccSnapshot[applied={T|T < 7298995083884253184}]
I20260620 18:47:01.657423 30021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:43766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:47:01.657531 30021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:43766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:47:01.658454 30021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Sending DeleteTablet for 3 replicas of tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f
I20260620 18:47:01.658769 30414 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:01 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:44020
I20260620 18:47:01.658813 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 30060
I20260620 18:47:01.658885 30282 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:01 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:35688
W20260620 18:47:01.663445 30007 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:34055 (error 108)
I20260620 18:47:01.663560 30510 tablet_replica.cc:330] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: stopping tablet replica
I20260620 18:47:01.663678 30510 raft_consensus.cc:2229] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:47:01.663703 30509 tablet_replica.cc:330] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: stopping tablet replica
I20260620 18:47:01.663697 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 30215
I20260620 18:47:01.663794 30510 raft_consensus.cc:2258] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:01.663791 30509 raft_consensus.cc:2229] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:47:01.663887 30509 raft_consensus.cc:2258] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:01.664359 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.664083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34055 (local address 127.0.0.1:33418)
0620 18:47:01.664271 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.664318 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260620 18:47:01.664956 30509 ts_tablet_manager.cc:1905] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 18:47:01.664484 30007 catalog_manager.cc:4584] TS 8ef88e1f9d674f4880390b6d8280e8a4 (127.15.77.65:34055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f: Network error: Client connection negotiation failed: client connection to 127.15.77.65:34055: connect: Connection refused (error 111)
I20260620 18:47:01.667018 30509 ts_tablet_manager.cc:1918] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:47:01.667099 30509 log.cc:1196] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/9bba3fb4fb6c4f4b8fab8da67b8d816f
I20260620 18:47:01.667418 30509 ts_tablet_manager.cc:1939] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Deleting consensus metadata
I20260620 18:47:01.668015 30009 catalog_manager.cc:4841] TS 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417): tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f (table test-table [id=f35dfe2bca9049229f37a0437726d32b]) successfully deleted
I20260620 18:47:01.669348 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 30351
I20260620 18:47:01.669368 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:01.669247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:39677 (local address 127.0.0.1:35690)
0620 18:47:01.669301 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.669340 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.66:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 18:47:01.674928 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 29989
I20260620 18:47:01.680509 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40491
--webserver_interface=127.15.77.126
--webserver_port=37085
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:01.763294 30512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:01.763540 30512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:01.763563 30512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:01.764680 30512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:47:01.764715 30512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:01.764729 30512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:01.764742 30512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:47:01.764753 30512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:47:01.766166 30512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=37085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:01.766373 30512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:01.766549 30512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:01.771469 30520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:01.771489 30518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:01.771627 30512 server_base.cc:1034] running on GCE node
W20260620 18:47:01.771430 30517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:01.772033 30512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:01.772361 30512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:01.773531 30512 hybrid_clock.cc:648] HybridClock initialized: now 1781981221773502 us; error 47 us; skew 500 ppm
Jun 20 18:47:01 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:01.776350 30512 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:47:01.776863 30512 webserver.cc:460] Webserver started at http://127.15.77.126:37085/ using document root <none> and password file <none>
I20260620 18:47:01.777110 30512 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:01.777182 30512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:01.778584 30512 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:01.779294 30527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.779529 30512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:01.779642 30512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "2fd617cf38b84ad58404b3602b86a606"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "044db5f9d1897dcb8e1c3bc4d88a2f9f"
server_key_iv: "e6cff40616b550c26e9bcd0b68181aab"
server_key_version: "encryptionkey@0"
I20260620 18:47:01.780036 30512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:01.799115 30512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:01.801708 30512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:01.801887 30512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:01.806213 30512 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:40491
I20260620 18:47:01.806257 30579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:40491 every 8 connection(s)
I20260620 18:47:01.806650 30512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:47:01.808336 30580 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:47:01.808972 30580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Bootstrap starting.
I20260620 18:47:01.815485 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 30512
I20260620 18:47:01.815861 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:34055
--local_ip_for_outbound_sockets=127.15.77.65
--tserver_master_addrs=127.15.77.126:40491
--webserver_port=41031
--webserver_interface=127.15.77.65
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:47:01.818171 30580 log.cc:824] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:01.820717 30580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:47:01.821043 30580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606: Bootstrap complete.
I20260620 18:47:01.823380 30580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:01.823628 30580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd617cf38b84ad58404b3602b86a606, State: Initialized, Role: FOLLOWER
I20260620 18:47:01.823805 30580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:01.823864 30580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:47:01.823925 30580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:47:01.823975 30580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 2 FOLLOWER]: Advancing to term 3
I20260620 18:47:01.824996 30580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:01.825081 30580 leader_election.cc:302] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fd617cf38b84ad58404b3602b86a606; no voters:
I20260620 18:47:01.825349 30580 leader_election.cc:288] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 18:47:01.825388 30584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 18:47:01.825505 30584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [term 3 LEADER]: Becoming Leader. State: Replica: 2fd617cf38b84ad58404b3602b86a606, State: Running, Role: LEADER
I20260620 18:47:01.825644 30580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:47:01.825683 30584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } }
I20260620 18:47:01.825968 30584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fd617cf38b84ad58404b3602b86a606. Latest consensus state: current_term: 3 leader_uuid: "2fd617cf38b84ad58404b3602b86a606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } } }
I20260620 18:47:01.826041 30584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: This master's current role is: LEADER
I20260620 18:47:01.826188 30586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2fd617cf38b84ad58404b3602b86a606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd617cf38b84ad58404b3602b86a606" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40491 } } }
I20260620 18:47:01.826263 30586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd617cf38b84ad58404b3602b86a606 [sys.catalog]: This master's current role is: LEADER
I20260620 18:47:01.826537 30590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:47:01.827450 30590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:47:01.827613 30590 catalog_manager.cc:1259] Loaded cluster ID: a07d44627b194021b58e364eb7755f11
I20260620 18:47:01.827816 30590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260620 18:47:01.828084 30590 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6251ce29ad google::LogMessage::Fail() at ??:0
@ 0x7f6251ce588a google::LogMessage::SendToLog() at ??:0
@ 0x7f6251ce24de google::LogMessage::Flush() at ??:0
@ 0x7f6251ce3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6259c030f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6259c0c8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6253bb00d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6253ba5954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6254a096db start_thread at ??:0
@ 0x7f62507e061f clone at ??:0
W20260620 18:47:01.914533 30582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:01.914834 30582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:01.914893 30582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:01.916162 30582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:01.916203 30582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:01.916231 30582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:47:01.917768 30582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:34055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=41031
--tserver_master_addrs=127.15.77.126:40491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:01.918009 30582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:01.918255 30582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:01.923261 30617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:01.923234 30619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:01.923592 30616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:01.923632 30582 server_base.cc:1034] running on GCE node
I20260620 18:47:01.923863 30582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:01.924141 30582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:01.925330 30582 hybrid_clock.cc:648] HybridClock initialized: now 1781981221925308 us; error 33 us; skew 500 ppm
Jun 20 18:47:01 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:01.928675 30582 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:47:01.929235 30582 webserver.cc:460] Webserver started at http://127.15.77.65:41031/ using document root <none> and password file <none>
I20260620 18:47:01.929497 30582 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:01.929561 30582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:01.931494 30582 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:47:01.932365 30626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:01.932626 30582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:01.932718 30582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "8ef88e1f9d674f4880390b6d8280e8a4"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "77b4f7eb66a1b9f0860af77bdfa46551"
server_key_iv: "a1cfaf63e514384d6fa901bb16961884"
server_key_version: "encryptionkey@0"
I20260620 18:47:01.933074 30582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:01.945150 30582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:01.949043 30582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:01.949369 30582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:01.949812 30582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:01.950522 30633 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 18:47:01.952129 30582 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 18:47:01.952188 30582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 18:47:01.952256 30582 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 18:47:01.953285 30582 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 18:47:01.953334 30582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 18:47:01.953410 30633 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Bootstrap starting.
I20260620 18:47:01.960829 30582 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:34055
I20260620 18:47:01.960927 30740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:34055 every 8 connection(s)
I20260620 18:47:01.961261 30582 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:47:01.961925 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 30582
I20260620 18:47:01.962250 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:39677
--local_ip_for_outbound_sockets=127.15.77.66
--tserver_master_addrs=127.15.77.126:40491
--webserver_port=37303
--webserver_interface=127.15.77.66
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:47:01.965093 30633 log.cc:824] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:01.968287 30633 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:47:01.968647 30633 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Bootstrap complete.
I20260620 18:47:01.969959 30633 ts_tablet_manager.cc:1397] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.008s
I20260620 18:47:01.971956 30633 raft_consensus.cc:348] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.972397 30633 raft_consensus.cc:729] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef88e1f9d674f4880390b6d8280e8a4, State: Initialized, Role: FOLLOWER
I20260620 18:47:01.972572 30633 consensus_queue.cc:260] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:01.972916 30633 ts_tablet_manager.cc:1428] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260620 18:47:02.049450 30744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:02.049666 30744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:02.049690 30744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:02.050882 30744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:02.050927 30744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:02.050954 30744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:47:02.052490 30744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:39677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=37303
--tserver_master_addrs=127.15.77.126:40491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:02.052744 30744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:02.052994 30744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:02.058487 30754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:02.058507 30751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:02.058507 30752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:02.058776 30744 server_base.cc:1034] running on GCE node
I20260620 18:47:02.059130 30744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:02.059522 30744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:02.060724 30744 hybrid_clock.cc:648] HybridClock initialized: now 1781981222060704 us; error 33 us; skew 500 ppm
Jun 20 18:47:02 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981222, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:02.063709 30744 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:47:02.064317 30744 webserver.cc:460] Webserver started at http://127.15.77.66:37303/ using document root <none> and password file <none>
I20260620 18:47:02.064590 30744 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:02.064659 30744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:02.066354 30744 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:47:02.067188 30761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:02.067433 30744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:02.067540 30744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "2a22635c494545e39a51e923fc1aba7d"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "bf1d3b50e376bde440175ce6c983f47a"
server_key_iv: "85155723fa320a4a5b182bd0e7af9a39"
server_key_version: "encryptionkey@0"
I20260620 18:47:02.067999 30744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:02.085114 30744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:02.087906 30744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:02.088101 30744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:02.088397 30744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:02.088961 30768 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 18:47:02.090474 30744 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 18:47:02.090539 30744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 18:47:02.090579 30744 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 18:47:02.091696 30744 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 18:47:02.091742 30744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:02.091830 30768 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Bootstrap starting.
W20260620 18:47:02.097100 30743 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:47:01.962050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:46351)
0620 18:47:01.962248 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:01.962288 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:01.962871 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:02.096854 (+133983us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":115,"threads_started":1}
W20260620 18:47:02.097342 30741 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:40491 (0 consecutive failures): Network error: Failed to ping master at 127.15.77.126:40491: Client connection negotiation failed: client connection to 127.15.77.126:40491: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 18:47:02.097711 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:02.097592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:60871)
0620 18:47:02.097646 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:02.097686 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 18:47:02.098073 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:02.097933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:48275)
0620 18:47:02.097998 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:02.098019 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 18:47:02.098184 30741 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 18:47:02.103744 30744 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:39677
I20260620 18:47:02.104053 30875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:39677 every 8 connection(s)
I20260620 18:47:02.104100 30768 log.cc:824] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:02.104290 30744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:47:02.106366 30768 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:47:02.106668 30768 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Bootstrap complete.
I20260620 18:47:02.107762 30768 ts_tablet_manager.cc:1397] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.012s
I20260620 18:47:02.108170 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:02.107716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:41089)
0620 18:47:02.107896 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:02.107988 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
W20260620 18:47:02.108428 30877 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:40491 (0 consecutive failures): Network error: Failed to ping master at 127.15.77.126:40491: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
I20260620 18:47:02.108776 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:02.108662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:51643)
0620 18:47:02.108716 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:02.108751 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 18:47:02.109107 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:02.108999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:33393)
0620 18:47:02.109050 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:02.109078 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 18:47:02.109257 30877 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 18:47:02.109301 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 30744
I20260620 18:47:02.109549 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:34417
--local_ip_for_outbound_sockets=127.15.77.67
--tserver_master_addrs=127.15.77.126:40491
--webserver_port=35589
--webserver_interface=127.15.77.67
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:47:02.109560 30768 raft_consensus.cc:348] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:02.109890 30768 raft_consensus.cc:729] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a22635c494545e39a51e923fc1aba7d, State: Initialized, Role: FOLLOWER
I20260620 18:47:02.110044 30768 consensus_queue.cc:260] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:02.110309 30768 ts_tablet_manager.cc:1428] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 18:47:02.110687 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:02.110584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:39821)
0620 18:47:02.110639 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:02.110665 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 18:47:02.111079 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:02.110981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:53897)
0620 18:47:02.111027 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:02.111053 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 18:47:02.194860 30881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:02.195068 30881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:02.195094 30881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:02.196238 30881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:02.196273 30881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:02.196300 30881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:47:02.197711 30881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:34417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=35589
--tserver_master_addrs=127.15.77.126:40491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:02.197906 30881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:02.198079 30881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:02.203131 30886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:02.203135 30887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:02.203104 30889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:02.203301 30881 server_base.cc:1034] running on GCE node
I20260620 18:47:02.203557 30881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:02.203837 30881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:02.205010 30881 hybrid_clock.cc:648] HybridClock initialized: now 1781981222205006 us; error 35 us; skew 500 ppm
Jun 20 18:47:02 dist-test-slave-tvl9 krb5kdc[29442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981222, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:02.207466 30881 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:47:02.207916 30881 webserver.cc:460] Webserver started at http://127.15.77.67:35589/ using document root <none> and password file <none>
I20260620 18:47:02.208156 30881 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:02.208226 30881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:02.209646 30881 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:47:02.210270 30896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:02.210454 30881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:47:02.210553 30881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "0af25ab364a746b0bf667ebb78ccf4e5"
format_stamp: "Formatted at 2026-06-20 18:47:00 on dist-test-slave-tvl9"
server_key: "fa927f4ec289fcfad53be9de6ccbfe21"
server_key_iv: "579efc1ad7973a193bbe8b7d6c57587b"
server_key_version: "encryptionkey@0"
I20260620 18:47:02.210937 30881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:02.223915 30881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:02.225844 30881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:02.226015 30881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:02.226266 30881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:02.226590 30881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:02.226644 30881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:02.226679 30881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:02.226707 30881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:02.232583 30881 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:34417
I20260620 18:47:02.232625 31009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:34417 every 8 connection(s)
I20260620 18:47:02.232960 30881 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:47:02.233989 31012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:02.233619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:35763)
0620 18:47:02.233775 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:02.233852 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
W20260620 18:47:02.234402 31010 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:40491 (0 consecutive failures): Network error: Failed to ping master at 127.15.77.126:40491: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
I20260620 18:47:02.234685 31012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:02.234597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:40387)
0620 18:47:02.234641 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:02.234668 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 18:47:02.234946 31012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:02.234892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:44447)
0620 18:47:02.234915 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:02.234934 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260620 18:47:02.235078 31010 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 18:47:02.235307 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 30881
I20260620 18:47:03.098809 31014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:03.098537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:42243)
0620 18:47:03.098697 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:03.098744 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260620 18:47:03.111852 31015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:03.111594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:42435)
0620 18:47:03.111753 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:03.111796 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":76,"threads_started":1}
I20260620 18:47:03.235728 31016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:03.235446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:44933)
0620 18:47:03.235582 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:03.235636 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260620 18:47:03.414273 31017 raft_consensus.cc:482] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:47:03.414410 31017 raft_consensus.cc:504] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:03.414824 31017 leader_election.cc:288] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417), 2a22635c494545e39a51e923fc1aba7d (127.15.77.66:39677)
Jun 20 18:47:03 dist-test-slave-tvl9 krb5kdc[29442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.66@KRBTEST.COM
Jun 20 18:47:03 dist-test-slave-tvl9 krb5kdc[29442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.67@KRBTEST.COM
I20260620 18:47:03.425884 31020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:03.415105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:46147 (local address 127.15.77.66:39677)
0620 18:47:03.415298 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:03.415307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:03.415387 (+ 80us) server_negotiation.cc:407] Connection header received
0620 18:47:03.415524 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:03.415527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:03.415582 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:03.415688 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:03.416552 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:03.417433 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:03.418533 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:03.418954 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:03.419063 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:03.421548 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:03.421554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:03.421555 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:03.421573 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:03.424042 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:03.424556 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:03.424560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:03.424560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:03.424693 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:03.424955 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:03.424956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:03.424957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:03.425224 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:03.425317 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:03.425630 (+ 313us) server_negotiation.cc:299] Negotiation successful
0620 18:47:03.425772 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260620 18:47:03.425871 31014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:03.414979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:39677 (local address 127.15.77.65:46147)
0620 18:47:03.415047 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:03.415059 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:03.415115 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:03.415760 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:03.415763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:03.416240 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:03.416431 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:03.416438 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:03.417560 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:03.417561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:03.418424 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:03.418427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:03.419043 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:03.419045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:03.419169 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:03.419885 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:03.419891 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:03.421417 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:03.424159 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:03.424166 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:03.424168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:03.424458 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:03.424769 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:03.424770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:03.424771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:03.424886 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:03.425313 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:03.425320 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:03.425442 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 18:47:03.425548 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 18:47:03.425764 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":43}
I20260620 18:47:03.425884 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:03.415115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:33417 (local address 127.15.77.67:34417)
0620 18:47:03.415241 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:03.415253 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:03.415292 (+ 39us) server_negotiation.cc:407] Connection header received
0620 18:47:03.415936 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:03.415939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:03.415987 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:03.416126 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:03.416570 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:03.417406 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:03.418583 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:03.418977 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:03.419106 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:03.421902 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:03.421906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:03.421906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:03.421919 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:03.424114 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:03.424580 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:03.424587 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:03.424588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:03.424681 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:03.424952 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:03.424953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:03.424954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:03.425224 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:03.425315 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:03.425618 (+ 303us) server_negotiation.cc:299] Negotiation successful
0620 18:47:03.425748 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":47,"threads_started":1}
I20260620 18:47:03.426311 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:03.415006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:34417 (local address 127.15.77.65:33417)
0620 18:47:03.415137 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:03.415149 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:03.415205 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:03.416121 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:03.416121 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:03.416330 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:03.416433 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:03.416438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:03.417518 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:03.417519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:03.418425 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:03.418428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:03.419083 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:03.419084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:03.419169 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:03.419827 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:03.419836 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:03.421788 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:03.424199 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:03.424200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:03.424201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:03.424464 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:03.424761 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:03.424761 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:03.424762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:03.424882 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:03.425307 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:03.425314 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:03.425441 (+ 127us) client_negotiation.cc:769] Sending connection context
0620 18:47:03.425551 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 18:47:03.425848 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":189,"thread_start_us":64,"threads_started":1}
I20260620 18:47:03.426491 30830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bba3fb4fb6c4f4b8fab8da67b8d816f" candidate_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2a22635c494545e39a51e923fc1aba7d" is_pre_election: true
I20260620 18:47:03.426679 30830 raft_consensus.cc:2454] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef88e1f9d674f4880390b6d8280e8a4 in term 1.
I20260620 18:47:03.427003 30629 leader_election.cc:302] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a22635c494545e39a51e923fc1aba7d, 8ef88e1f9d674f4880390b6d8280e8a4; no voters:
W20260620 18:47:03.427114 30630 leader_election.cc:341] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417): Not found: Tablet not found: 9bba3fb4fb6c4f4b8fab8da67b8d816f
I20260620 18:47:03.427140 31017 raft_consensus.cc:2781] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 18:47:03.427218 31017 raft_consensus.cc:482] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:47:03.427253 31017 raft_consensus.cc:3037] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 1 FOLLOWER]: Advancing to term 2
I20260620 18:47:03.428416 31017 raft_consensus.cc:504] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:03.428581 31017 leader_election.cc:288] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 2 election: Requested vote from peers 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417), 2a22635c494545e39a51e923fc1aba7d (127.15.77.66:39677)
I20260620 18:47:03.428762 30830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bba3fb4fb6c4f4b8fab8da67b8d816f" candidate_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2a22635c494545e39a51e923fc1aba7d"
W20260620 18:47:03.428828 30630 leader_election.cc:341] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417): Not found: Tablet not found: 9bba3fb4fb6c4f4b8fab8da67b8d816f
I20260620 18:47:03.428846 30830 raft_consensus.cc:3037] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 1 FOLLOWER]: Advancing to term 2
I20260620 18:47:03.430011 30830 raft_consensus.cc:2454] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef88e1f9d674f4880390b6d8280e8a4 in term 2.
I20260620 18:47:03.430195 30629 leader_election.cc:302] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a22635c494545e39a51e923fc1aba7d, 8ef88e1f9d674f4880390b6d8280e8a4; no voters: 0af25ab364a746b0bf667ebb78ccf4e5
I20260620 18:47:03.430286 31017 raft_consensus.cc:2781] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 18:47:03.430416 31017 raft_consensus.cc:686] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [term 2 LEADER]: Becoming Leader. State: Replica: 8ef88e1f9d674f4880390b6d8280e8a4, State: Running, Role: LEADER
I20260620 18:47:03.430526 31017 consensus_queue.cc:237] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:03.430781 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:03.430678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:34559)
0620 18:47:03.430720 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:03.430748 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260620 18:47:03.818574 30630 consensus_peers.cc:487] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 -> Peer 0af25ab364a746b0bf667ebb78ccf4e5 (127.15.77.67:34417): Couldn't send request to peer 0af25ab364a746b0bf667ebb78ccf4e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9bba3fb4fb6c4f4b8fab8da67b8d816f. This is attempt 1: this message will repeat every 5th retry.
I20260620 18:47:04.042259 30830 raft_consensus.cc:1261] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 2a22635c494545e39a51e923fc1aba7d [term 2 FOLLOWER]: Refusing update from remote peer 8ef88e1f9d674f4880390b6d8280e8a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260620 18:47:04.042727 31017 consensus_queue.cc:1035] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260620 18:47:04.043828 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:04.043602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:33505)
0620 18:47:04.043719 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:04.043789 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":44,"threads_started":1}
I20260620 18:47:04.044674 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:04.044477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:37763)
0620 18:47:04.044578 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:04.044627 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":68,"threads_started":1}
I20260620 18:47:04.044872 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:04.044783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:43973)
0620 18:47:04.044816 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:04.044845 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260620 18:47:04.207825 31030 ts_tablet_manager.cc:927] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Initiating tablet copy from peer 8ef88e1f9d674f4880390b6d8280e8a4 (127.15.77.65:34055)
I20260620 18:47:04.208066 31030 tablet_copy_client.cc:316] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: tablet copy: Beginning tablet copy session from remote peer at address 127.15.77.65:34055
Jun 20 18:47:04 dist-test-slave-tvl9 krb5kdc[29442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981222, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:47:04.215945 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:04.208400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34055 (local address 127.15.77.67:52555)
0620 18:47:04.208623 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:04.208659 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:04.208732 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:04.208913 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:04.208917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:04.209201 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:04.209266 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:04.209268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:04.210099 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:04.210100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:04.210705 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:04.210706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:04.211127 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:04.211128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:04.211189 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:04.211763 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:04.211767 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:04.213088 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:04.214883 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:04.214890 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:04.214891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:04.215083 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:04.215261 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:04.215261 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:04.215262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:04.215361 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:04.215655 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:04.215661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:04.215709 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 18:47:04.215774 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 18:47:04.215863 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":143,"threads_started":1}
I20260620 18:47:04.215998 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:04.208499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:52555 (local address 127.15.77.65:34055)
0620 18:47:04.208768 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:04.208773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:04.208785 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:47:04.208810 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:04.208811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:04.208825 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:04.208900 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:04.209354 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:04.210027 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:04.210774 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:04.211063 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:04.211118 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:04.213187 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:04.213196 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:04.213196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:04.213210 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:04.214806 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:04.215150 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:04.215153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:04.215153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:04.215206 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:04.215417 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:04.215418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:04.215418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:04.215591 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:04.215654 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:04.215816 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 18:47:04.215879 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":109,"threads_started":1}
I20260620 18:47:04.216454 30715 tablet_copy_service.cc:140] P 8ef88e1f9d674f4880390b6d8280e8a4: Received BeginTabletCopySession request for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f from peer 0af25ab364a746b0bf667ebb78ccf4e5 ({username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:52555)
I20260620 18:47:04.216542 30715 tablet_copy_service.cc:161] P 8ef88e1f9d674f4880390b6d8280e8a4: Beginning new tablet copy session on tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f from peer 0af25ab364a746b0bf667ebb78ccf4e5 at {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:52555: session id = 0af25ab364a746b0bf667ebb78ccf4e5-9bba3fb4fb6c4f4b8fab8da67b8d816f
I20260620 18:47:04.217005 30715 tablet_copy_source_session.cc:205] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 8ef88e1f9d674f4880390b6d8280e8a4: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 18:47:04.218014 31030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:04.220777 31030 tablet_copy_client.cc:791] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: tablet copy: Starting download of 0 data blocks...
I20260620 18:47:04.220933 31030 tablet_copy_client.cc:655] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: tablet copy: Starting download of 1 WAL segments...
I20260620 18:47:04.221748 31030 tablet_copy_client.cc:531] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 18:47:04.222414 31030 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Bootstrap starting.
I20260620 18:47:04.223824 31030 log.cc:824] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:04.226171 31030 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:47:04.226413 31030 tablet_bootstrap.cc:492] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Bootstrap complete.
I20260620 18:47:04.226557 31030 ts_tablet_manager.cc:1397] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 18:47:04.227684 31030 raft_consensus.cc:348] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:04.227900 31030 raft_consensus.cc:729] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af25ab364a746b0bf667ebb78ccf4e5, State: Initialized, Role: FOLLOWER
I20260620 18:47:04.228065 31030 consensus_queue.cc:260] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0af25ab364a746b0bf667ebb78ccf4e5" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 34417 } } peers { permanent_uuid: "2a22635c494545e39a51e923fc1aba7d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39677 } } peers { permanent_uuid: "8ef88e1f9d674f4880390b6d8280e8a4" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34055 } }
I20260620 18:47:04.228366 31030 ts_tablet_manager.cc:1428] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260620 18:47:04.228655 30715 tablet_copy_service.cc:342] P 8ef88e1f9d674f4880390b6d8280e8a4: Request end of tablet copy session 0af25ab364a746b0bf667ebb78ccf4e5-9bba3fb4fb6c4f4b8fab8da67b8d816f received from {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:52555
I20260620 18:47:04.228693 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:04.228560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:44213)
0620 18:47:04.228619 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:04.228658 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 18:47:04.228734 30715 tablet_copy_service.cc:433] P 8ef88e1f9d674f4880390b6d8280e8a4: ending tablet copy session 0af25ab364a746b0bf667ebb78ccf4e5-9bba3fb4fb6c4f4b8fab8da67b8d816f on tablet 9bba3fb4fb6c4f4b8fab8da67b8d816f with peer 0af25ab364a746b0bf667ebb78ccf4e5
I20260620 18:47:04.229038 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:04.228926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:59611)
0620 18:47:04.228978 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:04.229010 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 18:47:04.702159 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:04.701958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:44939)
0620 18:47:04.702065 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:04.702107 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":61,"threads_started":1}
I20260620 18:47:04.702153 30964 raft_consensus.cc:1203] T 9bba3fb4fb6c4f4b8fab8da67b8d816f P 0af25ab364a746b0bf667ebb78ccf4e5 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260620 18:47:04.702667 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:04.702515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:34907)
0620 18:47:04.702589 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:04.702632 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 18:47:05.045495 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:05.045292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:36127)
0620 18:47:05.045405 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:05.045446 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
I20260620 18:47:05.702854 31039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:05.702621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:36069)
0620 18:47:05.702749 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:05.702787 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260620 18:47:05.703418 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:05.703166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:55681)
0620 18:47:05.703283 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:05.703361 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260620 18:47:06.046168 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:06.045941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:42775)
0620 18:47:06.046081 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:06.046123 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260620 18:47:06.703850 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:06.703561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:42033)
0620 18:47:06.703739 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:06.703802 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":81,"threads_started":1}
I20260620 18:47:06.704207 31043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:06.703860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:36127)
0620 18:47:06.704091 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:06.704159 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
I20260620 18:47:07.046919 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:07.046662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:57695)
0620 18:47:07.046827 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:07.046872 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260620 18:47:07.704751 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:07.704438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:41361)
0620 18:47:07.704638 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:07.704694 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":82,"threads_started":1}
I20260620 18:47:07.705017 31046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:07.704776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:46089)
0620 18:47:07.704904 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:07.704963 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260620 18:47:07.705168 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:07.705033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:43963)
0620 18:47:07.705102 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:07.705135 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 18:47:08.047780 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:08.047560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:46441)
0620 18:47:08.047694 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:08.047731 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":57,"threads_started":1}
I20260620 18:47:08.705730 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:08.705489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:57937)
0620 18:47:08.705628 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:08.705671 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
I20260620 18:47:08.705807 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:08.705572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:38325)
0620 18:47:08.705716 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:08.705753 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260620 18:47:09.048585 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:09.048367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:56433)
0620 18:47:09.048497 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:09.048539 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260620 18:47:09.706658 31051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:09.706399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:40769)
0620 18:47:09.706550 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:09.706600 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260620 18:47:09.706681 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:09.706395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:37663)
0620 18:47:09.706590 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:09.706642 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":98,"threads_started":1}
I20260620 18:47:10.049299 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:10.049076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:42927)
0620 18:47:10.049215 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:10.049254 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260620 18:47:10.707427 31054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:10.707176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:34755)
0620 18:47:10.707333 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:10.707381 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260620 18:47:10.707453 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:10.707176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:48559)
0620 18:47:10.707367 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:10.707411 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":112,"threads_started":1}
I20260620 18:47:11.050110 31056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:11.049858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:45275)
0620 18:47:11.050017 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:11.050065 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260620 18:47:11.708146 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:11.707880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:38667)
0620 18:47:11.708033 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:11.708082 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":61,"threads_started":1}
I20260620 18:47:11.708344 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:11.708051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:41993)
0620 18:47:11.708192 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:11.708261 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260620 18:47:12.050882 31059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:12.050656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:38011)
0620 18:47:12.050794 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:12.050837 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260620 18:47:12.709107 31061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:12.708834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:60465)
0620 18:47:12.709001 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:12.709049 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260620 18:47:12.709156 31060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:12.708845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:43417)
0620 18:47:12.709021 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:12.709099 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
I20260620 18:47:13.051730 31062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:13.051447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:33497)
0620 18:47:13.051640 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:13.051683 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":108,"threads_started":1}
I20260620 18:47:13.710080 31063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:13.709838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:55953)
0620 18:47:13.709987 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:13.710034 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260620 18:47:13.710078 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:13.709838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:35107)
0620 18:47:13.710000 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:13.710034 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260620 18:47:14.052443 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:14.052221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:39375)
0620 18:47:14.052360 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:14.052396 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260620 18:47:14.710884 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:14.710627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:39135)
0620 18:47:14.710785 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:14.710831 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260620 18:47:14.710904 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:14.710644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:42213)
0620 18:47:14.710824 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:14.710864 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":86,"threads_started":1}
I20260620 18:47:15.053145 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:15.052909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:39709)
0620 18:47:15.053060 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:15.053097 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
I20260620 18:47:15.711702 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:15.711467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:42737)
0620 18:47:15.711606 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:15.711649 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260620 18:47:15.711740 31071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:15.711511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:60327)
0620 18:47:15.711668 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:15.711700 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260620 18:47:16.054075 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:16.053796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:52967)
0620 18:47:16.053975 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:16.054019 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260620 18:47:16.712453 31074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:16.712224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.65:47173)
0620 18:47:16.712361 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:16.712404 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260620 18:47:16.712468 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:16.712224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.67:55589)
0620 18:47:16.712391 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:16.712429 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":75,"threads_started":1}
I20260620 18:47:17.054875 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:17.054649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40491 (local address 127.15.77.66:53427)
0620 18:47:17.054784 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:17.054827 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260620 18:47:17.238785 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 30582
I20260620 18:47:17.244472 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 30744
I20260620 18:47:17.250424 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 30881
2026-06-20T18:47:17Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18229 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:47:17 dist-test-slave-tvl9 krb5kdc[31078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:47:17 dist-test-slave-tvl9 krb5kdc[31078](info): set up 2 sockets
Jun 20 18:47:17 dist-test-slave-tvl9 krb5kdc[31078](info): commencing operation
krb5kdc: starting...
W20260620 18:47:18.299500 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:47:18 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:47:18.417668 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:37883
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:37883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:18.501652 31091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:18.501922 31091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:18.501960 31091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:18.503060 31091 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 18:47:18.503093 31091 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 18:47:18.503106 31091 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 18:47:18.503118 31091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:47:18.503131 31091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:18.503144 31091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:18.503155 31091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:47:18.503173 31091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:47:18.504591 31091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:37883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:37883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:18.504778 31091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:18.504949 31091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:47:18.509935 31091 server_base.cc:1034] running on GCE node
W20260620 18:47:18.509876 31097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:18.509876 31099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:18.509881 31096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:18.510409 31091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:18.510720 31091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:18.511895 31091 hybrid_clock.cc:648] HybridClock initialized: now 1781981238511881 us; error 26 us; skew 500 ppm
Jun 20 18:47:18 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:18.514436 31091 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:47:18.514885 31091 webserver.cc:460] Webserver started at http://127.15.77.126:43217/ using document root <none> and password file <none>
I20260620 18:47:18.515144 31091 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:18.515251 31091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:18.515429 31091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:47:18.516561 31091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "e74d77882c5f4e5a8188a5829a9d8ccd"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "d3d9fc254cc3f951d6a055b7d63de6e7"
server_key_iv: "04866207aa8c3111f5c0f5efcd59590e"
server_key_version: "encryptionkey@0"
I20260620 18:47:18.516930 31091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "e74d77882c5f4e5a8188a5829a9d8ccd"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "d3d9fc254cc3f951d6a055b7d63de6e7"
server_key_iv: "04866207aa8c3111f5c0f5efcd59590e"
server_key_version: "encryptionkey@0"
I20260620 18:47:18.518294 31091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:47:18.519096 31106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:18.519366 31091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:18.519488 31091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "e74d77882c5f4e5a8188a5829a9d8ccd"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "d3d9fc254cc3f951d6a055b7d63de6e7"
server_key_iv: "04866207aa8c3111f5c0f5efcd59590e"
server_key_version: "encryptionkey@0"
I20260620 18:47:18.519575 31091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:18.537663 31091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:18.540153 31091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:18.540319 31091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:18.544842 31091 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:37883
I20260620 18:47:18.544876 31158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:37883 every 8 connection(s)
I20260620 18:47:18.545281 31091 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:47:18.546103 31159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:18.548408 31159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Bootstrap starting.
I20260620 18:47:18.548941 31159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:18.549185 31159 log.cc:824] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:18.549785 31159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: No bootstrap required, opened a new log
I20260620 18:47:18.551433 31159 raft_consensus.cc:348] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:47:18.551566 31159 raft_consensus.cc:374] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:18.551589 31159 raft_consensus.cc:729] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74d77882c5f4e5a8188a5829a9d8ccd, State: Initialized, Role: FOLLOWER
I20260620 18:47:18.551720 31159 consensus_queue.cc:260] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:47:18.551802 31159 raft_consensus.cc:388] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:47:18.551826 31159 raft_consensus.cc:482] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:47:18.551853 31159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:18.552335 31159 raft_consensus.cc:504] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:47:18.552417 31159 leader_election.cc:302] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e74d77882c5f4e5a8188a5829a9d8ccd; no voters:
I20260620 18:47:18.552604 31159 leader_election.cc:288] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:47:18.552661 31162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:47:18.552711 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 31091
I20260620 18:47:18.552803 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:47:18.552819 31162 raft_consensus.cc:686] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 1 LEADER]: Becoming Leader. State: Replica: e74d77882c5f4e5a8188a5829a9d8ccd, State: Running, Role: LEADER
I20260620 18:47:18.552881 31159 sys_catalog.cc:564] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:47:18.553046 15669 external_mini_cluster.cc:1428] Setting key f9f3d60f66e9d37bfc8a7f9dfc17cccd
I20260620 18:47:18.552902 31162 consensus_queue.cc:237] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:47:18.553551 31164 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } } }
I20260620 18:47:18.553537 31162 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e74d77882c5f4e5a8188a5829a9d8ccd. Latest consensus state: current_term: 1 leader_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } } }
I20260620 18:47:18.553714 31164 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: This master's current role is: LEADER
I20260620 18:47:18.553731 31162 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: This master's current role is: LEADER
I20260620 18:47:18.554639 31171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:47:18.555228 31171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:47:18.556862 31171 catalog_manager.cc:1347] Generated new cluster ID: 7f21d1fb44664cb483dca41b49fb68c0
I20260620 18:47:18.556941 31171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:47:18 dist-test-slave-tvl9 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:18.565995 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:18.553550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.15.77.126:37883)
0620 18:47:18.553990 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:18.554000 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:18.554044 (+ 44us) server_negotiation.cc:407] Connection header received
0620 18:47:18.554177 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:18.554184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:18.554274 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:18.554436 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:18.555401 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:18.556750 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:18.557870 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:18.558505 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:18.558606 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:18.561509 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:18.561513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:18.561514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:18.561527 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:18.564331 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:18.564722 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:18.564726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:18.564726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:18.564841 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:18.565044 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:18.565045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:18.565046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:18.565367 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:18.565520 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:18.565626 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 18:47:18.565772 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":73,"threads_started":1}
I20260620 18:47:18.588321 31171 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:47:18.589082 31171 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:47:18.596355 31171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Generated new TSK 0
I20260620 18:47:18.596601 31171 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:47:18.630470 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:37883
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:18.712283 31185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:18.712484 31185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:18.712507 31185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:18.713687 31185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:18.713728 31185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:18.713788 31185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:47:18.715359 31185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:18.715608 31185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:18.715850 31185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:47:18.720882 31185 server_base.cc:1034] running on GCE node
W20260620 18:47:18.720924 31191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:18.720851 31190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:18.720839 31193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:18.721309 31185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:18.721540 31185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:18.722676 31185 hybrid_clock.cc:648] HybridClock initialized: now 1781981238722668 us; error 30 us; skew 500 ppm
Jun 20 18:47:18 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:18.725276 31185 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:47:18.725721 31185 webserver.cc:460] Webserver started at http://127.15.77.65:36121/ using document root <none> and password file <none>
I20260620 18:47:18.725906 31185 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:18.725945 31185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:18.726034 31185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:47:18.727032 31185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "f0173537f098465b830c55067e2d2007"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "f927108b301d3996500fc5f5d0fd5879"
server_key_iv: "26890c3aa80cc763da5978e5a7690405"
server_key_version: "encryptionkey@0"
I20260620 18:47:18.727408 31185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "f0173537f098465b830c55067e2d2007"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "f927108b301d3996500fc5f5d0fd5879"
server_key_iv: "26890c3aa80cc763da5978e5a7690405"
server_key_version: "encryptionkey@0"
I20260620 18:47:18.728696 31185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:47:18.729382 31200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:18.729578 31185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:47:18.729671 31185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "f0173537f098465b830c55067e2d2007"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "f927108b301d3996500fc5f5d0fd5879"
server_key_iv: "26890c3aa80cc763da5978e5a7690405"
server_key_version: "encryptionkey@0"
I20260620 18:47:18.729725 31185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:18.743219 31185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:18.745018 31185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:18.745182 31185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:18.745428 31185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:18.745754 31185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:18.745798 31185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:18.745846 31185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:18.745873 31185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:18.751849 31185 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:34631
I20260620 18:47:18.751899 31313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:34631 every 8 connection(s)
I20260620 18:47:18.752228 31185 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:47:18.755501 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 31185
I20260620 18:47:18.755613 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:47:18.755816 15669 external_mini_cluster.cc:1428] Setting key d30d3aa11a3713bc7a25efdffad77253
Jun 20 18:47:18 dist-test-slave-tvl9 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:18.762543 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:18.752940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:57371 (local address 127.15.77.126:37883)
0620 18:47:18.752983 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:18.752985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:18.753218 (+ 233us) server_negotiation.cc:407] Connection header received
0620 18:47:18.753781 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:18.753783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:18.753799 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:18.753862 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:18.754887 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:18.755518 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:18.757007 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:18.757209 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:18.757259 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:18.759616 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:18.759618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:18.759618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:18.759632 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:18.761102 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:18.761553 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:18.761555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:18.761555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:18.761617 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:18.761877 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:18.761878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:18.761879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:18.761999 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:18.762075 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:18.762356 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 18:47:18.762449 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 18:47:18.762733 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:18.752911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:57371)
0620 18:47:18.753055 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:18.753124 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:18.753677 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:18.753936 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:18.753940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:18.754594 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:18.754789 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:18.754794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:18.755635 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:18.755637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:18.756909 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:18.756912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:18.757293 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:18.757294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:18.757445 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:18.757967 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:18.757977 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:18.759517 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:18.761230 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:18.761237 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:18.761240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:18.761475 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:18.761692 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:18.761692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:18.761693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:18.761810 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:18.762068 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:18.762075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:18.762223 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 18:47:18.762333 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 18:47:18.762495 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260620 18:47:18.763280 31314 heartbeater.cc:347] Connected to a master server at 127.15.77.126:37883
I20260620 18:47:18.763401 31314 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:18.763586 31314 heartbeater.cc:511] Master 127.15.77.126:37883 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:47:18.764178 31123 ts_manager.cc:194] Registered new tserver with Master: f0173537f098465b830c55067e2d2007 (127.15.77.65:34631)
I20260620 18:47:18.764780 31123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:57371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:47:18.780347 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:37883
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:18.862279 31319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:18.862471 31319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:18.862494 31319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:18.863582 31319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:18.863621 31319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:18.863646 31319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:47:18.865087 31319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:18.865272 31319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:18.865440 31319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:18.870347 31324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:18.870474 31319 server_base.cc:1034] running on GCE node
W20260620 18:47:18.870416 31325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:18.870347 31327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:18.870837 31319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:18.871081 31319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:18.872248 31319 hybrid_clock.cc:648] HybridClock initialized: now 1781981238872235 us; error 41 us; skew 500 ppm
Jun 20 18:47:18 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:18.874670 31319 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:47:18.875094 31319 webserver.cc:460] Webserver started at http://127.15.77.66:40623/ using document root <none> and password file <none>
I20260620 18:47:18.875394 31319 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:18.875444 31319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:18.875591 31319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:47:18.876621 31319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "fe18e6f18f1b470fb672eb9eaa3945ca"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "ac99d7169c3b788d8f7b67048eed8e40"
server_key_iv: "b6b940a6b85e09d9449bf0f70ac88688"
server_key_version: "encryptionkey@0"
I20260620 18:47:18.876987 31319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "fe18e6f18f1b470fb672eb9eaa3945ca"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "ac99d7169c3b788d8f7b67048eed8e40"
server_key_iv: "b6b940a6b85e09d9449bf0f70ac88688"
server_key_version: "encryptionkey@0"
I20260620 18:47:18.878279 31319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:47:18.878983 31334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:18.879164 31319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:47:18.879256 31319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "fe18e6f18f1b470fb672eb9eaa3945ca"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "ac99d7169c3b788d8f7b67048eed8e40"
server_key_iv: "b6b940a6b85e09d9449bf0f70ac88688"
server_key_version: "encryptionkey@0"
I20260620 18:47:18.879369 31319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:18.910887 31319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:18.912814 31319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:18.912981 31319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:18.913231 31319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:18.913570 31319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:18.913635 31319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:18.913681 31319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:18.913710 31319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:18.919559 31319 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:41593
I20260620 18:47:18.919611 31447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:41593 every 8 connection(s)
I20260620 18:47:18.919929 31319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:47:18.925585 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 31319
I20260620 18:47:18.925690 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:47:18.925848 15669 external_mini_cluster.cc:1428] Setting key 86b3fd3cb61152a7a5514d2ea4c7a46a
Jun 20 18:47:18 dist-test-slave-tvl9 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:18.929469 31450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:18.920560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:47721)
0620 18:47:18.920747 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:18.920784 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:18.921265 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:18.921476 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:18.921480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:18.922059 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:18.922261 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:18.922266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:18.923064 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:18.923065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:18.923933 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:18.923935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:18.924265 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:18.924266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:18.924363 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:18.924886 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:18.924895 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:18.926297 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:18.928235 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:18.928242 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:18.928243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:18.928450 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:18.928653 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:18.928654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:18.928654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:18.928747 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:18.928988 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:18.928991 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:18.929101 (+ 110us) client_negotiation.cc:769] Sending connection context
0620 18:47:18.929191 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 18:47:18.929307 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
I20260620 18:47:18.929466 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:18.920603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:47721 (local address 127.15.77.126:37883)
0620 18:47:18.920686 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:18.920689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:18.920875 (+ 186us) server_negotiation.cc:407] Connection header received
0620 18:47:18.921350 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:18.921351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:18.921361 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:18.921416 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:18.922338 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:18.922983 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:18.924005 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:18.924195 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:18.924250 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:18.926374 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:18.926376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:18.926376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:18.926383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:18.928144 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:18.928521 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:18.928522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:18.928522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:18.928584 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:18.928812 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:18.928813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:18.928813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:18.928931 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:18.928986 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:18.929231 (+ 245us) server_negotiation.cc:299] Negotiation successful
0620 18:47:18.929325 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 18:47:18.929911 31448 heartbeater.cc:347] Connected to a master server at 127.15.77.126:37883
I20260620 18:47:18.930015 31448 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:18.930217 31448 heartbeater.cc:511] Master 127.15.77.126:37883 requested a full tablet report, sending...
I20260620 18:47:18.930658 31123 ts_manager.cc:194] Registered new tserver with Master: fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593)
I20260620 18:47:18.931033 31123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:47721
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:47:18.951766 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:37883
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:19.032523 31453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:19.032811 31453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:19.032876 31453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:19.034030 31453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:19.034070 31453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:19.034152 31453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:47:19.035651 31453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:19.035898 31453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:19.036151 31453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:19.041239 31458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:19.041247 31459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:19.041368 31453 server_base.cc:1034] running on GCE node
W20260620 18:47:19.041473 31461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:19.041699 31453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:19.041896 31453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:19.043061 31453 hybrid_clock.cc:648] HybridClock initialized: now 1781981239043042 us; error 28 us; skew 500 ppm
Jun 20 18:47:19 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981239, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:19.045513 31453 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:47:19.045895 31453 webserver.cc:460] Webserver started at http://127.15.77.67:41533/ using document root <none> and password file <none>
I20260620 18:47:19.046123 31453 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:19.046192 31453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:19.046346 31453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:47:19.047470 31453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "dfeed14f4cf54e0391861a65e5c11b52"
format_stamp: "Formatted at 2026-06-20 18:47:19 on dist-test-slave-tvl9"
server_key: "debfde5a65fbf57792eb726df7a724c2"
server_key_iv: "a32fce1ae4d467154f72e8e76d81047b"
server_key_version: "encryptionkey@0"
I20260620 18:47:19.047827 31453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "dfeed14f4cf54e0391861a65e5c11b52"
format_stamp: "Formatted at 2026-06-20 18:47:19 on dist-test-slave-tvl9"
server_key: "debfde5a65fbf57792eb726df7a724c2"
server_key_iv: "a32fce1ae4d467154f72e8e76d81047b"
server_key_version: "encryptionkey@0"
I20260620 18:47:19.049116 31453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:47:19.049847 31468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:19.050021 31453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:47:19.050120 31453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "dfeed14f4cf54e0391861a65e5c11b52"
format_stamp: "Formatted at 2026-06-20 18:47:19 on dist-test-slave-tvl9"
server_key: "debfde5a65fbf57792eb726df7a724c2"
server_key_iv: "a32fce1ae4d467154f72e8e76d81047b"
server_key_version: "encryptionkey@0"
I20260620 18:47:19.050201 31453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:19.065703 31453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:19.068277 31453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:19.068446 31453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:19.068696 31453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:19.069017 31453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:19.069073 31453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:19.069118 31453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:19.069142 31453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:19.075066 31453 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:37685
I20260620 18:47:19.075110 31581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:37685 every 8 connection(s)
I20260620 18:47:19.075480 31453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:47:19.076418 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 31453
I20260620 18:47:19.076520 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:47:19.076694 15669 external_mini_cluster.cc:1428] Setting key f495f4704fd1df5db8c15847dd8d0ee8
Jun 20 18:47:19 dist-test-slave-tvl9 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981239, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:19.084916 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.076179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:54163 (local address 127.15.77.126:37883)
0620 18:47:19.076242 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:19.076245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:19.076448 (+ 203us) server_negotiation.cc:407] Connection header received
0620 18:47:19.077106 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:19.077107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:19.077121 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:19.077175 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:19.078121 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.078629 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.079788 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.080105 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.080178 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.082404 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:19.082405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:19.082405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:19.082412 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:19.083815 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:19.084172 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:19.084173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:19.084173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:19.084220 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:19.084426 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:19.084426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:19.084426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:19.084523 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:19.084570 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:19.084791 (+ 221us) server_negotiation.cc:299] Negotiation successful
0620 18:47:19.084853 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 18:47:19.085147 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.076163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:54163)
0620 18:47:19.076319 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:19.076355 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:19.077006 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:19.077247 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:19.077251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:19.077816 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:19.078015 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.078019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.078732 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.078733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.079716 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.079719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.080187 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.080188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.080349 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.080972 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:19.080980 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:19.082324 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:19.083887 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:19.083893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:19.083895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:19.084107 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:19.084276 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:19.084277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:19.084277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:19.084371 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:19.084575 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:19.084579 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:19.084688 (+ 109us) client_negotiation.cc:769] Sending connection context
0620 18:47:19.084767 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 18:47:19.084888 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":106,"threads_started":1}
I20260620 18:47:19.085722 31582 heartbeater.cc:347] Connected to a master server at 127.15.77.126:37883
I20260620 18:47:19.085817 31582 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:19.086011 31582 heartbeater.cc:511] Master 127.15.77.126:37883 requested a full tablet report, sending...
I20260620 18:47:19.086395 31123 ts_manager.cc:194] Registered new tserver with Master: dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685)
I20260620 18:47:19.086786 31123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:54163
I20260620 18:47:19.088266 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:47:19.095486 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.089584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.15.77.126:37883)
0620 18:47:19.089783 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:19.089785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:19.089880 (+ 95us) server_negotiation.cc:407] Connection header received
0620 18:47:19.089955 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:19.089955 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:19.089966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:19.090021 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:19.090569 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.091045 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.091848 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.092051 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.092127 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.093222 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:19.093223 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:19.093224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:19.093231 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:19.094293 (+ 1062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:19.094632 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:19.094633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:19.094633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:19.094689 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:19.094890 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:19.094891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:19.094891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:19.095038 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:19.095140 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:19.095294 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 18:47:19.095388 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260620 18:47:19.097152 31123 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:47:19.098716 31123 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:47:19.104722 31595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.101028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34631 (local address 127.0.0.1:55190)
0620 18:47:19.101186 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:19.101227 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:19.101290 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:19.101546 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:19.101548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:19.101550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:19.101598 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.101599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.102671 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.102672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.103569 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.103571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.104375 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.104376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.104475 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.104488 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 18:47:19.104589 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 18:47:19.104651 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":45,"threads_started":1}
I20260620 18:47:19.104729 31597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.101231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.15.77.65:34631)
0620 18:47:19.101408 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:19.101413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:19.101422 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:47:19.101454 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:19.101455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:19.101467 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:19.101529 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:19.101677 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.102559 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.103658 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.104287 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.104376 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.104437 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:19.104608 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 18:47:19.104651 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260620 18:47:19.105824 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.101042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:37685 (local address 127.0.0.1:45558)
0620 18:47:19.101909 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:19.101920 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:19.101976 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:19.102249 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:19.102250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:19.102252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:19.102291 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.102293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.103523 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.103524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.104661 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.104663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.105578 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.105579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.105649 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.105654 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:47:19.105717 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 18:47:19.105759 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260620 18:47:19.105825 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.101262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.15.77.67:37685)
0620 18:47:19.101440 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:19.101445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:19.101975 (+ 530us) server_negotiation.cc:407] Connection header received
0620 18:47:19.102046 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:19.102049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:19.102064 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:19.102165 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:19.102370 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.103428 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.104762 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.105491 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.105567 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.105619 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:19.105734 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 18:47:19.105767 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":76,"threads_started":1}
I20260620 18:47:19.105692 31248 tablet_service.cc:1467] Processing CreateTablet for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61 (DEFAULT_TABLE table=test-table [id=2006bc6c5e454f86a501114e11155c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:19.106105 31248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:19.106290 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.101029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:41593 (local address 127.0.0.1:40328)
0620 18:47:19.101394 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:19.101405 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:19.101443 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:19.102043 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:19.102044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:19.102045 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:19.102082 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.102084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.103419 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.103420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.104660 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.104663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.106067 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.106069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.106130 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.106134 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:47:19.106191 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 18:47:19.106228 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":34,"thread_start_us":78,"threads_started":1}
I20260620 18:47:19.106526 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.101212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40328 (local address 127.15.77.66:41593)
0620 18:47:19.101654 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:19.101658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:19.101668 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:47:19.101707 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:19.101710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:19.101745 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:19.101818 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:19.102160 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.103284 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.104762 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.105491 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.105567 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.105620 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:19.106252 (+ 632us) server_negotiation.cc:299] Negotiation successful
0620 18:47:19.106295 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":84,"threads_started":1}
I20260620 18:47:19.106484 31516 tablet_service.cc:1467] Processing CreateTablet for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61 (DEFAULT_TABLE table=test-table [id=2006bc6c5e454f86a501114e11155c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:19.106793 31516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:19.108083 31382 tablet_service.cc:1467] Processing CreateTablet for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61 (DEFAULT_TABLE table=test-table [id=2006bc6c5e454f86a501114e11155c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:19.108426 31382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:19.109145 31599 tablet_bootstrap.cc:492] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Bootstrap starting.
I20260620 18:47:19.109829 31599 tablet_bootstrap.cc:654] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:19.109838 31600 tablet_bootstrap.cc:492] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: Bootstrap starting.
I20260620 18:47:19.110528 31600 tablet_bootstrap.cc:654] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:19.110543 31599 log.cc:824] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:19.111105 31600 log.cc:824] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:19.111428 31599 tablet_bootstrap.cc:492] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: No bootstrap required, opened a new log
I20260620 18:47:19.111574 31599 ts_tablet_manager.cc:1397] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:47:19.111884 31600 tablet_bootstrap.cc:492] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: No bootstrap required, opened a new log
I20260620 18:47:19.111958 31600 ts_tablet_manager.cc:1397] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260620 18:47:19.113582 31599 raft_consensus.cc:348] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:19.113879 31599 raft_consensus.cc:374] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:19.113919 31603 tablet_bootstrap.cc:492] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: Bootstrap starting.
I20260620 18:47:19.113842 31600 raft_consensus.cc:348] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:19.113966 31599 raft_consensus.cc:729] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0173537f098465b830c55067e2d2007, State: Initialized, Role: FOLLOWER
I20260620 18:47:19.114001 31600 raft_consensus.cc:374] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:19.114039 31600 raft_consensus.cc:729] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfeed14f4cf54e0391861a65e5c11b52, State: Initialized, Role: FOLLOWER
I20260620 18:47:19.114121 31599 consensus_queue.cc:260] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:19.114151 31600 consensus_queue.cc:260] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:19.114413 31600 ts_tablet_manager.cc:1428] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 18:47:19.114420 31599 ts_tablet_manager.cc:1428] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 18:47:19.114511 31582 heartbeater.cc:503] Master 127.15.77.126:37883 was elected leader, sending a full tablet report...
I20260620 18:47:19.114521 31314 heartbeater.cc:503] Master 127.15.77.126:37883 was elected leader, sending a full tablet report...
I20260620 18:47:19.114683 31603 tablet_bootstrap.cc:654] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:19.115231 31603 log.cc:824] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:19.115933 31603 tablet_bootstrap.cc:492] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: No bootstrap required, opened a new log
I20260620 18:47:19.115988 31603 ts_tablet_manager.cc:1397] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 18:47:19.117298 31603 raft_consensus.cc:348] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:19.117425 31603 raft_consensus.cc:374] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:19.117448 31603 raft_consensus.cc:729] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe18e6f18f1b470fb672eb9eaa3945ca, State: Initialized, Role: FOLLOWER
I20260620 18:47:19.117609 31603 consensus_queue.cc:260] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:19.117889 31603 ts_tablet_manager.cc:1428] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:47:19.117936 31448 heartbeater.cc:503] Master 127.15.77.126:37883 was elected leader, sending a full tablet report...
W20260620 18:47:19.170588 31449 tablet.cc:2369] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 18:47:19.253000 31315 tablet.cc:2369] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 18:47:19.326022 31583 tablet.cc:2369] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:47:19.507498 31604 raft_consensus.cc:482] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:47:19.507639 31604 raft_consensus.cc:504] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:19.508025 31604 leader_election.cc:288] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593), f0173537f098465b830c55067e2d2007 (127.15.77.65:34631)
I20260620 18:47:19.510923 31597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.508320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:33205 (local address 127.15.77.65:34631)
0620 18:47:19.508381 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:19.508384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:19.508400 (+ 16us) server_negotiation.cc:407] Connection header received
0620 18:47:19.508435 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:19.508437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:19.508447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:19.508515 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:19.508698 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.509225 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.510210 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.510563 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.510653 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.510662 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:19.510806 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 18:47:19.510848 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 18:47:19.510936 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.508460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:51331 (local address 127.15.77.66:41593)
0620 18:47:19.508526 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:19.508528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:19.508537 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:47:19.508561 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:19.508562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:19.508571 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:19.508644 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:19.508825 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.509288 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.510215 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.510560 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.510653 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.510662 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:19.510833 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 18:47:19.510868 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 18:47:19.510938 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.508252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34631 (local address 127.15.77.67:33205)
0620 18:47:19.508314 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:19.508327 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:19.508372 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:19.508542 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:19.508543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:19.508545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:19.508620 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.508622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.509314 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.509315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.510104 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.510106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.510649 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.510650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.510737 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.510740 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:47:19.510782 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 18:47:19.510847 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":43}
I20260620 18:47:19.510964 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.508252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:41593 (local address 127.15.77.67:51331)
0620 18:47:19.508424 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:19.508435 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:19.508501 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:19.508673 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:19.508674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:19.508676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:19.508750 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.508751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.509377 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.509377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.510117 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:19.510119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.510656 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.510658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:19.510741 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.510744 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:47:19.510813 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 18:47:19.510849 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":20,"thread_start_us":93,"threads_started":1}
I20260620 18:47:19.511281 31402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fdc669e09840ffa6e0fe3aa2ac0d61" candidate_uuid: "dfeed14f4cf54e0391861a65e5c11b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" is_pre_election: true
I20260620 18:47:19.511471 31402 raft_consensus.cc:2454] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfeed14f4cf54e0391861a65e5c11b52 in term 0.
I20260620 18:47:19.511422 31268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fdc669e09840ffa6e0fe3aa2ac0d61" candidate_uuid: "dfeed14f4cf54e0391861a65e5c11b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0173537f098465b830c55067e2d2007" is_pre_election: true
I20260620 18:47:19.511592 31268 raft_consensus.cc:2454] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfeed14f4cf54e0391861a65e5c11b52 in term 0.
I20260620 18:47:19.511745 31469 leader_election.cc:302] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfeed14f4cf54e0391861a65e5c11b52, fe18e6f18f1b470fb672eb9eaa3945ca; no voters:
I20260620 18:47:19.511891 31604 raft_consensus.cc:2781] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:47:19.511979 31604 raft_consensus.cc:482] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:47:19.512017 31604 raft_consensus.cc:3037] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:19.512715 31604 raft_consensus.cc:504] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:19.512871 31604 leader_election.cc:288] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [CANDIDATE]: Term 1 election: Requested vote from peers fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593), f0173537f098465b830c55067e2d2007 (127.15.77.65:34631)
I20260620 18:47:19.513043 31268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fdc669e09840ffa6e0fe3aa2ac0d61" candidate_uuid: "dfeed14f4cf54e0391861a65e5c11b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0173537f098465b830c55067e2d2007"
I20260620 18:47:19.513053 31402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30fdc669e09840ffa6e0fe3aa2ac0d61" candidate_uuid: "dfeed14f4cf54e0391861a65e5c11b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca"
I20260620 18:47:19.513126 31402 raft_consensus.cc:3037] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:19.513134 31268 raft_consensus.cc:3037] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:19.514008 31268 raft_consensus.cc:2454] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfeed14f4cf54e0391861a65e5c11b52 in term 1.
I20260620 18:47:19.514008 31402 raft_consensus.cc:2454] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfeed14f4cf54e0391861a65e5c11b52 in term 1.
I20260620 18:47:19.514211 31470 leader_election.cc:302] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfeed14f4cf54e0391861a65e5c11b52, f0173537f098465b830c55067e2d2007; no voters:
I20260620 18:47:19.514305 31604 raft_consensus.cc:2781] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:47:19.514482 31604 raft_consensus.cc:686] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 1 LEADER]: Becoming Leader. State: Replica: dfeed14f4cf54e0391861a65e5c11b52, State: Running, Role: LEADER
I20260620 18:47:19.514585 31604 consensus_queue.cc:237] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:19.515460 31122 catalog_manager.cc:5495] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67). New cstate: current_term: 1 leader_uuid: "dfeed14f4cf54e0391861a65e5c11b52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } health_report { overall_health: HEALTHY } } }
I20260620 18:47:19.546512 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.543935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45566 (local address 127.15.77.67:37685)
0620 18:47:19.544000 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:19.544006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:19.544015 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:47:19.544049 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:19.544050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:19.544067 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:19.544126 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:47:19.544242 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.544709 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.545746 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:19.545931 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:19.545980 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:19.546159 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:47:19.546254 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:47:19.546310 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:19.546379 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 18:47:19.546397 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 18:47:19.548415 31268 raft_consensus.cc:1261] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Refusing update from remote peer dfeed14f4cf54e0391861a65e5c11b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:47:19.548422 31402 raft_consensus.cc:1261] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca [term 1 FOLLOWER]: Refusing update from remote peer dfeed14f4cf54e0391861a65e5c11b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:47:19.548820 31609 consensus_queue.cc:1035] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:47:19.548952 31604 consensus_queue.cc:1035] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:47:19.551407 31612 mvcc.cc:205] Tried to move back new op lower bound from 7298995157187137536 to 7298995157052182528. Current Snapshot: MvccSnapshot[applied={T|T < 7298995157187137536}]
I20260620 18:47:19.551831 31613 mvcc.cc:205] Tried to move back new op lower bound from 7298995157187137536 to 7298995157052182528. Current Snapshot: MvccSnapshot[applied={T|T < 7298995157187137536}]
I20260620 18:47:19.553825 31123 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:47:19.553915 31123 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:47:19.554667 31123 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Sending DeleteTablet for 3 replicas of tablet 30fdc669e09840ffa6e0fe3aa2ac0d61
I20260620 18:47:19.555013 31248 tablet_service.cc:1514] Processing DeleteTablet for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:19 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:55190
I20260620 18:47:19.555015 31382 tablet_service.cc:1514] Processing DeleteTablet for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:19 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:40328
I20260620 18:47:19.555243 31621 tablet_replica.cc:330] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: stopping tablet replica
I20260620 18:47:19.555292 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 31185
I20260620 18:47:19.555287 31516 tablet_service.cc:1514] Processing DeleteTablet for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:19 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:45558
I20260620 18:47:19.555840 31623 tablet_replica.cc:330] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: stopping tablet replica
I20260620 18:47:19.555918 31623 raft_consensus.cc:2229] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:47:19.555363 31621 raft_consensus.cc:2229] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:47:19.556022 31623 raft_consensus.cc:2258] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:19.556172 31621 raft_consensus.cc:2258] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:19.556571 31623 ts_tablet_manager.cc:1905] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:47:19.556583 31621 ts_tablet_manager.cc:1905] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:47:19.558414 31621 ts_tablet_manager.cc:1918] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:47:19.558425 31623 ts_tablet_manager.cc:1918] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:47:19.558485 31621 log.cc:1196] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/30fdc669e09840ffa6e0fe3aa2ac0d61
I20260620 18:47:19.558485 31623 log.cc:1196] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/30fdc669e09840ffa6e0fe3aa2ac0d61
I20260620 18:47:19.558743 31623 ts_tablet_manager.cc:1939] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P dfeed14f4cf54e0391861a65e5c11b52: Deleting consensus metadata
I20260620 18:47:19.558748 31621 ts_tablet_manager.cc:1939] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P fe18e6f18f1b470fb672eb9eaa3945ca: Deleting consensus metadata
I20260620 18:47:19.559120 31107 catalog_manager.cc:4841] TS fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593): tablet 30fdc669e09840ffa6e0fe3aa2ac0d61 (table test-table [id=2006bc6c5e454f86a501114e11155c87]) successfully deleted
I20260620 18:47:19.559120 31109 catalog_manager.cc:4841] TS dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685): tablet 30fdc669e09840ffa6e0fe3aa2ac0d61 (table test-table [id=2006bc6c5e454f86a501114e11155c87]) successfully deleted
W20260620 18:47:19.562552 31108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:34631 (error 108)
I20260620 18:47:19.563071 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:19.562938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34631 (local address 127.0.0.1:55198)
0620 18:47:19.562995 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:19.563031 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:34631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 18:47:19.563146 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 31319
W20260620 18:47:19.566566 31108 catalog_manager.cc:4584] TS f0173537f098465b830c55067e2d2007 (127.15.77.65:34631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61: Network error: Client connection negotiation failed: client connection to 127.15.77.65:34631: connect: Connection refused (error 111)
I20260620 18:47:19.569985 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 31453
I20260620 18:47:19.574786 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 31091
I20260620 18:47:39.580339 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:37883
--webserver_interface=127.15.77.126
--webserver_port=43217
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:37883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:47:39.663199 31625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:39.663450 31625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:39.663473 31625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:39.664718 31625 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 18:47:39.664767 31625 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 18:47:39.664780 31625 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 18:47:39.664791 31625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:47:39.664804 31625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:39.664817 31625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:39.664877 31625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:47:39.664894 31625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:47:39.666414 31625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:37883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:37883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=43217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:39.666693 31625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:39.666941 31625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:39.676103 31632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:39.676100 31631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:39.676147 31625 server_base.cc:1034] running on GCE node
W20260620 18:47:39.676139 31634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:39.676759 31625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:39.677058 31625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:39.678220 31625 hybrid_clock.cc:648] HybridClock initialized: now 1781981259678208 us; error 31 us; skew 500 ppm
Jun 20 18:47:39 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:39.685531 31625 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:47:39.685961 31625 webserver.cc:460] Webserver started at http://127.15.77.126:43217/ using document root <none> and password file <none>
I20260620 18:47:39.686172 31625 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:39.686256 31625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:39.687738 31625 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:47:39.688483 31641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:39.688717 31625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:47:39.688819 31625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "e74d77882c5f4e5a8188a5829a9d8ccd"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "d3d9fc254cc3f951d6a055b7d63de6e7"
server_key_iv: "04866207aa8c3111f5c0f5efcd59590e"
server_key_version: "encryptionkey@0"
I20260620 18:47:39.689311 31625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:39.721154 31625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:39.723129 31625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:39.723287 31625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:39.727257 31625 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:37883
I20260620 18:47:39.727319 31693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:37883 every 8 connection(s)
I20260620 18:47:39.727753 31625 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:47:39.729293 31694 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:47:39.730034 31694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Bootstrap starting.
I20260620 18:47:39.736132 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 31625
I20260620 18:47:39.736459 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:34631
--local_ip_for_outbound_sockets=127.15.77.65
--tserver_master_addrs=127.15.77.126:37883
--webserver_port=36121
--webserver_interface=127.15.77.65
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:47:39.739180 31694 log.cc:824] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:39.741241 31694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:47:39.741557 31694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Bootstrap complete.
I20260620 18:47:39.743929 31694 raft_consensus.cc:348] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:47:39.744182 31694 raft_consensus.cc:729] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74d77882c5f4e5a8188a5829a9d8ccd, State: Initialized, Role: FOLLOWER
I20260620 18:47:39.744390 31694 consensus_queue.cc:260] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:47:39.744488 31694 raft_consensus.cc:388] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:47:39.744547 31694 raft_consensus.cc:482] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:47:39.744596 31694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 1 FOLLOWER]: Advancing to term 2
I20260620 18:47:39.745594 31694 raft_consensus.cc:504] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:47:39.745723 31694 leader_election.cc:302] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e74d77882c5f4e5a8188a5829a9d8ccd; no voters:
I20260620 18:47:39.745923 31694 leader_election.cc:288] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 18:47:39.745959 31698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 2 FOLLOWER]: Leader election won for term 2
I20260620 18:47:39.746073 31698 raft_consensus.cc:686] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 2 LEADER]: Becoming Leader. State: Replica: e74d77882c5f4e5a8188a5829a9d8ccd, State: Running, Role: LEADER
I20260620 18:47:39.746205 31694 sys_catalog.cc:564] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:47:39.746234 31698 consensus_queue.cc:237] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:47:39.746618 31700 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } } }
I20260620 18:47:39.746703 31700 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: This master's current role is: LEADER
I20260620 18:47:39.746924 31699 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e74d77882c5f4e5a8188a5829a9d8ccd. Latest consensus state: current_term: 2 leader_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } } }
I20260620 18:47:39.747035 31699 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: This master's current role is: LEADER
I20260620 18:47:39.747565 31704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:47:39.748466 31704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:47:39.748570 31704 catalog_manager.cc:1259] Loaded cluster ID: 7f21d1fb44664cb483dca41b49fb68c0
I20260620 18:47:39.748605 31704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:47:39.749876 31704 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:47:39.750072 31704 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Loaded TSK: 0
I20260620 18:47:39.762207 31704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Generated new TSK 1
I20260620 18:47:39.762321 31704 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 18:47:39.824958 31696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:39.825162 31696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:39.825186 31696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:39.826382 31696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:39.826428 31696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:39.826455 31696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:47:39.828033 31696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:34631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=36121
--tserver_master_addrs=127.15.77.126:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:39.828330 31696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:39.828603 31696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:39.833674 31724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:39.833658 31721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:39.833796 31722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:39.834043 31696 server_base.cc:1034] running on GCE node
I20260620 18:47:39.834244 31696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:39.834544 31696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:39.835790 31696 hybrid_clock.cc:648] HybridClock initialized: now 1781981259835777 us; error 29 us; skew 500 ppm
Jun 20 18:47:39 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:39.838511 31696 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:47:39.839035 31696 webserver.cc:460] Webserver started at http://127.15.77.65:36121/ using document root <none> and password file <none>
I20260620 18:47:39.839349 31696 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:39.839449 31696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:39.841195 31696 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:47:39.842051 31731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:39.842252 31696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:39.842345 31696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "f0173537f098465b830c55067e2d2007"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "f927108b301d3996500fc5f5d0fd5879"
server_key_iv: "26890c3aa80cc763da5978e5a7690405"
server_key_version: "encryptionkey@0"
I20260620 18:47:39.842782 31696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:39.864260 31696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:39.866765 31696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:39.866909 31696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:39.867149 31696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:39.867640 31738 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 18:47:39.869159 31696 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 18:47:39.869220 31696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 18:47:39.869254 31696 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 18:47:39.870087 31696 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 18:47:39.870131 31696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 18:47:39.870270 31738 tablet_bootstrap.cc:492] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Bootstrap starting.
I20260620 18:47:39.877125 31696 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:34631
I20260620 18:47:39.877318 31845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:34631 every 8 connection(s)
I20260620 18:47:39.877704 31696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:47:39.881201 31738 log.cc:824] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:39.882424 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 31696
I20260620 18:47:39.882704 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:41593
--local_ip_for_outbound_sockets=127.15.77.66
--tserver_master_addrs=127.15.77.126:37883
--webserver_port=40623
--webserver_interface=127.15.77.66
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:47:39.883971 31738 tablet_bootstrap.cc:492] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:47:39.884280 31738 tablet_bootstrap.cc:492] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Bootstrap complete.
I20260620 18:47:39.885545 31738 ts_tablet_manager.cc:1397] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.011s
Jun 20 18:47:39 dist-test-slave-tvl9 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:39.887810 31738 raft_consensus.cc:348] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:39.888329 31738 raft_consensus.cc:729] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0173537f098465b830c55067e2d2007, State: Initialized, Role: FOLLOWER
I20260620 18:47:39.888985 31738 consensus_queue.cc:260] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } }
I20260620 18:47:39.889701 31738 ts_tablet_manager.cc:1428] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 18:47:39.892320 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:39.878580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:33207 (local address 127.15.77.126:37883)
0620 18:47:39.878808 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:39.878819 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:39.879700 (+ 881us) server_negotiation.cc:407] Connection header received
0620 18:47:39.880610 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:39.880617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:39.880673 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:39.880806 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:39.881690 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:39.883060 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:39.884658 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:39.885268 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:39.885375 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:39.888154 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:39.888159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:39.888159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:39.888172 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:39.890280 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:39.890838 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:39.890841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:39.890842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:39.890927 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:39.891248 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:39.891249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:39.891249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:39.891571 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:39.891639 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:39.892035 (+ 396us) server_negotiation.cc:299] Negotiation successful
0620 18:47:39.892179 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":90,"threads_started":1}
I20260620 18:47:39.892464 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:39.878471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:33207)
0620 18:47:39.879472 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:39.879529 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:39.880352 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:39.880877 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:39.880881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:39.881288 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:39.881549 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:39.881555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:39.883206 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:39.883207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:39.884548 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:39.884551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:39.885479 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:39.885480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:39.885629 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:39.886275 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:39.886286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:39.888030 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:39.890381 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:39.890388 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:39.890390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:39.890733 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:39.891008 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:39.891009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:39.891009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:39.891172 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:39.891644 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:39.891652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:39.891852 (+ 200us) client_negotiation.cc:769] Sending connection context
0620 18:47:39.891978 (+ 126us) client_negotiation.cc:240] Negotiation successful
0620 18:47:39.892212 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"thread_start_us":92,"threads_started":1}
I20260620 18:47:39.893313 31846 heartbeater.cc:347] Connected to a master server at 127.15.77.126:37883
I20260620 18:47:39.893424 31846 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:39.893631 31846 heartbeater.cc:511] Master 127.15.77.126:37883 requested a full tablet report, sending...
I20260620 18:47:39.894364 31658 ts_manager.cc:194] Registered new tserver with Master: f0173537f098465b830c55067e2d2007 (127.15.77.65:34631)
I20260620 18:47:39.895445 31658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:33207
I20260620 18:47:39.896405 31846 heartbeater.cc:503] Master 127.15.77.126:37883 was elected leader, sending a full tablet report...
Jun 20 18:47:39 dist-test-slave-tvl9 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.65@KRBTEST.COM
I20260620 18:47:39.903314 31853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:39.895010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34631 (local address 127.0.0.1:51542)
0620 18:47:39.895424 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:39.895464 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:39.895529 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:39.895901 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:39.895905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:39.896133 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:39.896180 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:39.896182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:39.897232 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:39.897233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:39.897806 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:39.897808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:39.898286 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:39.898288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:39.898371 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:39.898861 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:39.898866 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:39.900286 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:39.902109 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:39.902116 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:39.902118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:39.902321 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:39.902547 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:39.902548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:39.902548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:39.902631 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:39.902925 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:39.902933 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:39.902986 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 18:47:39.903088 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 18:47:39.903213 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":104,"threads_started":1}
I20260620 18:47:39.903347 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:39.895088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51542 (local address 127.15.77.65:34631)
0620 18:47:39.895652 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:39.895657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:39.895666 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:47:39.895696 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:39.895699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:39.895712 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:39.895776 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:39.896271 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:39.897135 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:39.897896 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:39.898190 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:39.898281 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:39.900387 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:39.900394 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:39.900395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:39.900401 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:39.901959 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:39.902409 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:39.902412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:39.902413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:39.902478 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:39.902696 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:39.902697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:39.902697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:39.902847 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:39.902934 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:39.903124 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 18:47:39.903216 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":81,"threads_started":1}
I20260620 18:47:39.903955 31780 tablet_service.cc:1514] Processing DeleteTablet for tablet 30fdc669e09840ffa6e0fe3aa2ac0d61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:19 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:51542
I20260620 18:47:39.904574 31855 tablet_replica.cc:330] stopping tablet replica
I20260620 18:47:39.904696 31855 raft_consensus.cc:2229] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:47:39.904790 31855 raft_consensus.cc:2258] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:39.905102 31855 ts_tablet_manager.cc:1905] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:47:39.907188 31855 ts_tablet_manager.cc:1918] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:47:39.907276 31855 log.cc:1196] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal/wals/30fdc669e09840ffa6e0fe3aa2ac0d61
I20260620 18:47:39.907531 31855 ts_tablet_manager.cc:1939] T 30fdc669e09840ffa6e0fe3aa2ac0d61 P f0173537f098465b830c55067e2d2007: Deleting consensus metadata
I20260620 18:47:39.908058 31643 catalog_manager.cc:4841] TS f0173537f098465b830c55067e2d2007 (127.15.77.65:34631): tablet 30fdc669e09840ffa6e0fe3aa2ac0d61 (table test-table [id=2006bc6c5e454f86a501114e11155c87]) successfully deleted
W20260620 18:47:39.969795 31851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:39.970019 31851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:39.970043 31851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:39.971133 31851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:39.971169 31851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:39.971194 31851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:47:39.972651 31851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:41593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=40623
--tserver_master_addrs=127.15.77.126:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:39.972838 31851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:39.973009 31851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:39.977789 31860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:39.977837 31861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:39.977789 31863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:39.978087 31851 server_base.cc:1034] running on GCE node
I20260620 18:47:39.978259 31851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:39.978457 31851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:39.979595 31851 hybrid_clock.cc:648] HybridClock initialized: now 1781981259979581 us; error 32 us; skew 500 ppm
Jun 20 18:47:39 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:39.986207 31851 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:47:39.986690 31851 webserver.cc:460] Webserver started at http://127.15.77.66:40623/ using document root <none> and password file <none>
I20260620 18:47:39.986925 31851 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:39.987004 31851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:39.988471 31851 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:39.989207 31870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:39.989384 31851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:39.989485 31851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "fe18e6f18f1b470fb672eb9eaa3945ca"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "ac99d7169c3b788d8f7b67048eed8e40"
server_key_iv: "b6b940a6b85e09d9449bf0f70ac88688"
server_key_version: "encryptionkey@0"
I20260620 18:47:39.989880 31851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:40.004117 31851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:40.006448 31851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:40.006609 31851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:40.006837 31851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:40.007210 31851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:40.007269 31851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:40.007330 31851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:40.007360 31851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:40.013278 31851 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:41593
I20260620 18:47:40.013325 31983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:41593 every 8 connection(s)
I20260620 18:47:40.013657 31851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:47:40.019265 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 31851
I20260620 18:47:40.019574 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:37685
--local_ip_for_outbound_sockets=127.15.77.67
--tserver_master_addrs=127.15.77.126:37883
--webserver_port=41533
--webserver_interface=127.15.77.67
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 18:47:40 dist-test-slave-tvl9 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:40.023077 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.014346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:60225 (local address 127.15.77.126:37883)
0620 18:47:40.014414 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:40.014416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:40.014620 (+ 204us) server_negotiation.cc:407] Connection header received
0620 18:47:40.015111 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:40.015112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:40.015123 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:40.015174 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:40.016147 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.016656 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.017594 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.017788 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.017837 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.020098 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:40.020100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:40.020100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:40.020109 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:40.021643 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:40.022135 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:40.022136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:40.022136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:40.022187 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:40.022463 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:40.022464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:40.022464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:40.022576 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:40.022625 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:40.022945 (+ 320us) server_negotiation.cc:299] Negotiation successful
0620 18:47:40.023014 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 18:47:40.023424 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.014340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:60225)
0620 18:47:40.014489 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:40.014526 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:40.015017 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:40.015227 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:40.015230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:40.015851 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:40.016049 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.016054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.016739 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.016741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.017523 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.017525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.017860 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.017861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.017957 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.018493 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:40.018501 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:40.020001 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:40.021725 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:40.021733 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:40.021735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:40.022059 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:40.022252 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:40.022253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:40.022253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:40.022400 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:40.022640 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:40.022646 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:40.022812 (+ 166us) client_negotiation.cc:769] Sending connection context
0620 18:47:40.022925 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 18:47:40.023105 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260620 18:47:40.024019 31984 heartbeater.cc:347] Connected to a master server at 127.15.77.126:37883
I20260620 18:47:40.024127 31984 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:40.024325 31984 heartbeater.cc:511] Master 127.15.77.126:37883 requested a full tablet report, sending...
I20260620 18:47:40.024713 31658 ts_manager.cc:194] Registered new tserver with Master: fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593)
I20260620 18:47:40.025085 31658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:60225
W20260620 18:47:40.122102 31987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:47:40.122355 31987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:47:40.122395 31987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:47:40.123649 31987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:47:40.123709 31987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:47:40.123752 31987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:47:40.125226 31987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:37685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=41533
--tserver_master_addrs=127.15.77.126:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:47:40.125502 31987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:47:40.125741 31987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:47:40.130750 31995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:40.130756 31993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:47:40.130746 31992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:47:40.131119 31987 server_base.cc:1034] running on GCE node
I20260620 18:47:40.131378 31987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:47:40.131659 31987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:47:40.132826 31987 hybrid_clock.cc:648] HybridClock initialized: now 1781981260132816 us; error 39 us; skew 500 ppm
Jun 20 18:47:40 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981260, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:47:40.135232 31987 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:47:40.135725 31987 webserver.cc:460] Webserver started at http://127.15.77.67:41533/ using document root <none> and password file <none>
I20260620 18:47:40.135969 31987 fs_manager.cc:359] Metadata directory not provided
I20260620 18:47:40.136054 31987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:47:40.137436 31987 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:47:40.138000 32002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:40.138149 31987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:47:40.138255 31987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "dfeed14f4cf54e0391861a65e5c11b52"
format_stamp: "Formatted at 2026-06-20 18:47:19 on dist-test-slave-tvl9"
server_key: "debfde5a65fbf57792eb726df7a724c2"
server_key_iv: "a32fce1ae4d467154f72e8e76d81047b"
server_key_version: "encryptionkey@0"
I20260620 18:47:40.138605 31987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:47:40.153229 31987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:47:40.155419 31987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:47:40.155594 31987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:47:40.155846 31987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:47:40.156172 31987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:47:40.156231 31987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:40.156266 31987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:47:40.156293 31987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:47:40.162492 31987 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:37685
I20260620 18:47:40.162535 32115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:37685 every 8 connection(s)
I20260620 18:47:40.162932 31987 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:47:40.165485 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 31987
Jun 20 18:47:40 dist-test-slave-tvl9 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981260, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:47:40.173151 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.165954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57808 (local address 127.15.77.126:37883)
0620 18:47:40.166075 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:40.166077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:40.166147 (+ 70us) server_negotiation.cc:407] Connection header received
0620 18:47:40.166220 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:40.166221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:40.166232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:40.166291 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:40.167060 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.167597 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.168591 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.168802 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.168885 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.170341 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:40.170342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:40.170343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:40.170351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:40.171917 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:40.172301 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:40.172301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:40.172302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:40.172352 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:40.172578 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:40.172579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:40.172579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:40.172711 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:40.172896 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:40.172941 (+ 45us) server_negotiation.cc:299] Negotiation successful
0620 18:47:40.173018 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260620 18:47:40.174027 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.163749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:35273 (local address 127.15.77.126:37883)
0620 18:47:40.163797 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:40.163799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:40.164046 (+ 247us) server_negotiation.cc:407] Connection header received
0620 18:47:40.164522 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:40.164523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:40.164535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:40.164583 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:40.165743 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.166382 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.167754 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.167932 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.167978 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.170710 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:40.170711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:40.170712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:40.170719 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:40.172458 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:40.172940 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:40.172941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:40.172941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:40.172999 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:40.173326 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:40.173327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:40.173327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:40.173449 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:40.173514 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:40.173838 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 18:47:40.173937 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:47:40.174255 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.163762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:35273)
0620 18:47:40.163914 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:40.163952 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:40.164434 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:40.164655 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:40.164659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:40.165239 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:47:40.165584 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.165593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.166481 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.166482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.167667 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.167671 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.168012 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.168013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.168165 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.168770 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:47:40.168779 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:47:40.170616 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:47:40.172551 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:40.172560 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:40.172563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:40.172868 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:40.173099 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:47:40.173100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:47:40.173100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:47:40.173244 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:47:40.173511 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:47:40.173519 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:47:40.173683 (+ 164us) client_negotiation.cc:769] Sending connection context
0620 18:47:40.173819 (+ 136us) client_negotiation.cc:240] Negotiation successful
0620 18:47:40.173992 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260620 18:47:40.174692 32116 heartbeater.cc:347] Connected to a master server at 127.15.77.126:37883
I20260620 18:47:40.174808 32116 heartbeater.cc:464] Registering TS with master...
I20260620 18:47:40.174993 32116 heartbeater.cc:511] Master 127.15.77.126:37883 requested a full tablet report, sending...
I20260620 18:47:40.175407 31658 ts_manager.cc:194] Registered new tserver with Master: dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685)
I20260620 18:47:40.175788 31658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:35273
I20260620 18:47:40.185067 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:47:40.187940 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.185441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57820 (local address 127.15.77.126:37883)
0620 18:47:40.185524 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:40.185526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:40.185643 (+ 117us) server_negotiation.cc:407] Connection header received
0620 18:47:40.185699 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:40.185700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:40.185736 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:40.185804 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:47:40.185902 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.186392 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.187194 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.187460 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.187512 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.187660 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:47:40.187706 (+ 46us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0620 18:47:40.187867 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57820: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260620 18:47:40.188014 31849 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57820: token expired
W20260620 18:47:40.188025 31585 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.77.126:37883)
I20260620 18:47:40.194523 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.188264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.15.77.126:37883)
0620 18:47:40.188332 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:40.188334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:40.188343 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:47:40.188363 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:40.188364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:40.188372 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:40.188420 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:47:40.188906 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.189403 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.190106 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.190290 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.190365 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.191659 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:47:40.191661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:47:40.191661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:47:40.191668 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:47:40.193319 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:40.193702 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:40.193703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:40.193704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:40.193789 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:47:40.194065 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:47:40.194065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:47:40.194066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:47:40.194180 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:47:40.194265 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:40.194379 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 18:47:40.194454 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 18:47:40.195868 31658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:47:40.197559 31658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:47:40.199800 31780 tablet_service.cc:1467] Processing CreateTablet for tablet 5368eebc10074fff8602cda978c81c9b (DEFAULT_TABLE table=test-table [id=cbfe3190b36342faa1f2f3e94bbe88db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:40.200017 31780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5368eebc10074fff8602cda978c81c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:40.201527 31738 tablet_bootstrap.cc:492] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: Bootstrap starting.
I20260620 18:47:40.201875 31738 tablet_bootstrap.cc:654] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:40.202913 31738 tablet_bootstrap.cc:492] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: No bootstrap required, opened a new log
I20260620 18:47:40.202996 31738 ts_tablet_manager.cc:1397] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 18:47:40.203174 31738 raft_consensus.cc:348] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:47:40.203259 31738 raft_consensus.cc:374] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:40.203322 31738 raft_consensus.cc:729] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0173537f098465b830c55067e2d2007, State: Initialized, Role: FOLLOWER
I20260620 18:47:40.203388 31738 consensus_queue.cc:260] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:47:40.203495 31738 ts_tablet_manager.cc:1428] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 18:47:40.204556 31853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.199550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:41593 (local address 127.0.0.1:35146)
0620 18:47:40.199615 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:40.199627 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:40.199665 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:40.200074 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:40.200075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:40.200077 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:40.200135 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.200136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.201210 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.201212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.202450 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.202453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.204278 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.204279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.204351 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.204372 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 18:47:40.204432 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 18:47:40.204482 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 18:47:40.204566 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.199694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.15.77.66:41593)
0620 18:47:40.199903 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:40.199908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:40.199919 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:47:40.199966 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:40.199969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:40.199985 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:40.200085 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:40.200201 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.201122 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.202562 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.204182 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.204280 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.204349 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:40.204449 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 18:47:40.204486 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
I20260620 18:47:40.205034 32124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.199550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:37685 (local address 127.0.0.1:47968)
0620 18:47:40.201613 (+ 2063us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:40.201627 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:40.201683 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:40.201899 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:40.201899 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:40.201901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:40.201956 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.201958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.202961 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.202962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.203874 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.203876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.204793 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.204794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.204857 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.204862 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:47:40.204916 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 18:47:40.204959 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2046,"mutex_wait_us":367,"thread_start_us":46,"threads_started":1}
I20260620 18:47:40.205235 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.199750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47968 (local address 127.15.77.67:37685)
0620 18:47:40.199955 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:40.199959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:40.201751 (+ 1792us) server_negotiation.cc:407] Connection header received
0620 18:47:40.201799 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:40.201801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:40.201815 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:40.201903 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:40.202006 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.202896 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.203975 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.204705 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.204789 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.204843 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:40.205014 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 18:47:40.205076 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":117,"threads_started":1}
I20260620 18:47:40.205385 31918 tablet_service.cc:1467] Processing CreateTablet for tablet 5368eebc10074fff8602cda978c81c9b (DEFAULT_TABLE table=test-table [id=cbfe3190b36342faa1f2f3e94bbe88db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:40.205739 31918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5368eebc10074fff8602cda978c81c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:40.205893 32050 tablet_service.cc:1467] Processing CreateTablet for tablet 5368eebc10074fff8602cda978c81c9b (DEFAULT_TABLE table=test-table [id=cbfe3190b36342faa1f2f3e94bbe88db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:47:40.206182 32050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5368eebc10074fff8602cda978c81c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:47:40.208803 32126 tablet_bootstrap.cc:492] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Bootstrap starting.
I20260620 18:47:40.209182 32127 tablet_bootstrap.cc:492] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: Bootstrap starting.
I20260620 18:47:40.209540 32126 tablet_bootstrap.cc:654] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:40.209839 32127 tablet_bootstrap.cc:654] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: Neither blocks nor log segments found. Creating new log.
I20260620 18:47:40.210201 32126 log.cc:824] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:40.210459 32127 log.cc:824] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: Log is configured to *not* fsync() on all Append() calls
I20260620 18:47:40.210970 32126 tablet_bootstrap.cc:492] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: No bootstrap required, opened a new log
I20260620 18:47:40.211026 32126 ts_tablet_manager.cc:1397] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:47:40.211225 32127 tablet_bootstrap.cc:492] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: No bootstrap required, opened a new log
I20260620 18:47:40.211306 32127 ts_tablet_manager.cc:1397] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:47:40.212637 32126 raft_consensus.cc:348] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:47:40.212810 32126 raft_consensus.cc:374] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:40.212857 32126 raft_consensus.cc:729] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe18e6f18f1b470fb672eb9eaa3945ca, State: Initialized, Role: FOLLOWER
I20260620 18:47:40.212749 32127 raft_consensus.cc:348] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:47:40.212929 32127 raft_consensus.cc:374] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:47:40.212975 32127 raft_consensus.cc:729] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfeed14f4cf54e0391861a65e5c11b52, State: Initialized, Role: FOLLOWER
I20260620 18:47:40.212980 32126 consensus_queue.cc:260] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:47:40.213106 32127 consensus_queue.cc:260] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:47:40.213299 32126 ts_tablet_manager.cc:1428] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:47:40.213320 31984 heartbeater.cc:503] Master 127.15.77.126:37883 was elected leader, sending a full tablet report...
I20260620 18:47:40.213406 32127 ts_tablet_manager.cc:1428] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:47:40.213421 32116 heartbeater.cc:503] Master 127.15.77.126:37883 was elected leader, sending a full tablet report...
W20260620 18:47:40.264189 31985 tablet.cc:2369] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:47:40.334985 31852 raft_consensus.cc:482] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:47:40.335134 31852 raft_consensus.cc:504] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:47:40.335598 31852 leader_election.cc:288] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593), dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685)
I20260620 18:47:40.338511 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.335818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:40341 (local address 127.15.77.66:41593)
0620 18:47:40.335875 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:40.335878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:40.335888 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:47:40.335985 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:40.335986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:40.336004 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:40.336070 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:40.336255 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.336747 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.337764 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.338131 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.338220 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.338231 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:40.338395 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 18:47:40.338434 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 18:47:40.338526 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.335737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:41593 (local address 127.15.77.65:40341)
0620 18:47:40.335811 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:40.335824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:40.335881 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:40.336105 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:40.336106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:40.336108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:40.336181 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.336182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.336844 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.336846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.337660 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.337662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.338218 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.338219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.338298 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.338316 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 18:47:40.338388 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 18:47:40.338436 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":26}
I20260620 18:47:40.338527 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.335914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:45791 (local address 127.15.77.67:37685)
0620 18:47:40.335973 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:40.335976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:40.336007 (+ 31us) server_negotiation.cc:407] Connection header received
0620 18:47:40.336090 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:40.336091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:40.336100 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:40.336158 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:47:40.336320 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.336787 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.337757 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.338154 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.338243 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.338253 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:40.338415 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 18:47:40.338452 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 18:47:40.338608 32132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.335776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:37685 (local address 127.15.77.65:45791)
0620 18:47:40.335945 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:40.335957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:40.336020 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:40.336163 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:47:40.336164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:47:40.336165 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:47:40.336243 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.336245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.336878 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.336880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.337655 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:47:40.337657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.338256 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.338257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:47:40.338337 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.338340 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:47:40.338409 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 18:47:40.338453 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":4,"thread_start_us":94,"threads_started":1}
I20260620 18:47:40.338913 32070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5368eebc10074fff8602cda978c81c9b" candidate_uuid: "f0173537f098465b830c55067e2d2007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfeed14f4cf54e0391861a65e5c11b52" is_pre_election: true
I20260620 18:47:40.339092 32070 raft_consensus.cc:2454] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0173537f098465b830c55067e2d2007 in term 0.
I20260620 18:47:40.339075 31938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5368eebc10074fff8602cda978c81c9b" candidate_uuid: "f0173537f098465b830c55067e2d2007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" is_pre_election: true
I20260620 18:47:40.339263 31938 raft_consensus.cc:2454] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0173537f098465b830c55067e2d2007 in term 0.
I20260620 18:47:40.339339 31734 leader_election.cc:302] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfeed14f4cf54e0391861a65e5c11b52, f0173537f098465b830c55067e2d2007; no voters:
I20260620 18:47:40.339546 31852 raft_consensus.cc:2781] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:47:40.339632 31852 raft_consensus.cc:482] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:47:40.339668 31852 raft_consensus.cc:3037] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:40.340365 31852 raft_consensus.cc:504] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:47:40.340523 31852 leader_election.cc:288] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 1 election: Requested vote from peers fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593), dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685)
I20260620 18:47:40.340664 31938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5368eebc10074fff8602cda978c81c9b" candidate_uuid: "f0173537f098465b830c55067e2d2007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca"
I20260620 18:47:40.340740 31938 raft_consensus.cc:3037] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:40.340709 32070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5368eebc10074fff8602cda978c81c9b" candidate_uuid: "f0173537f098465b830c55067e2d2007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfeed14f4cf54e0391861a65e5c11b52"
I20260620 18:47:40.340816 32070 raft_consensus.cc:3037] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:47:40.341456 31938 raft_consensus.cc:2454] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0173537f098465b830c55067e2d2007 in term 1.
I20260620 18:47:40.341640 31732 leader_election.cc:302] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0173537f098465b830c55067e2d2007, fe18e6f18f1b470fb672eb9eaa3945ca; no voters:
I20260620 18:47:40.341648 32070 raft_consensus.cc:2454] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0173537f098465b830c55067e2d2007 in term 1.
I20260620 18:47:40.341743 31852 raft_consensus.cc:2781] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:47:40.341923 31852 raft_consensus.cc:686] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 LEADER]: Becoming Leader. State: Replica: f0173537f098465b830c55067e2d2007, State: Running, Role: LEADER
I20260620 18:47:40.342015 31852 consensus_queue.cc:237] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:47:40.342815 31658 catalog_manager.cc:5495] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0173537f098465b830c55067e2d2007 (127.15.77.65). New cstate: current_term: 1 leader_uuid: "f0173537f098465b830c55067e2d2007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } health_report { overall_health: HEALTHY } } }
I20260620 18:47:40.348865 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.346550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.15.77.65:34631)
0620 18:47:40.346631 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 18:47:40.346634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:47:40.346675 (+ 41us) server_negotiation.cc:407] Connection header received
0620 18:47:40.346715 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:47:40.346716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:47:40.346731 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:47:40.346792 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:47:40.346891 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.347447 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.348225 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:47:40.348420 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:47:40.348474 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:47:40.348606 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:47:40.348704 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:47:40.348757 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 18:47:40.348792 (+ 35us) server_negotiation.cc:299] Negotiation successful
0620 18:47:40.348809 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 18:47:40.350399 31938 raft_consensus.cc:1261] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca [term 1 FOLLOWER]: Refusing update from remote peer f0173537f098465b830c55067e2d2007: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:47:40.350412 32070 raft_consensus.cc:1261] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52 [term 1 FOLLOWER]: Refusing update from remote peer f0173537f098465b830c55067e2d2007: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:47:40.350739 32133 consensus_queue.cc:1035] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:47:40.350880 31852 consensus_queue.cc:1035] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:47:40.353610 32136 mvcc.cc:205] Tried to move back new op lower bound from 7298995242392965120 to 7298995242361552896. Current Snapshot: MvccSnapshot[applied={T|T < 7298995242392965120 or (T in {7298995242392965120})}]
I20260620 18:47:40.354065 32137 mvcc.cc:205] Tried to move back new op lower bound from 7298995242392965120 to 7298995242361552896. Current Snapshot: MvccSnapshot[applied={T|T < 7298995242392965120}]
I20260620 18:47:40.355641 31658 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:47:40.355724 31658 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:47:40.356480 31658 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Sending DeleteTablet for 3 replicas of tablet 5368eebc10074fff8602cda978c81c9b
I20260620 18:47:40.356794 31780 tablet_service.cc:1514] Processing DeleteTablet for tablet 5368eebc10074fff8602cda978c81c9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:40 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:51542
I20260620 18:47:40.356791 31918 tablet_service.cc:1514] Processing DeleteTablet for tablet 5368eebc10074fff8602cda978c81c9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:40 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:35146
I20260620 18:47:40.356885 31855 tablet_replica.cc:330] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: stopping tablet replica
I20260620 18:47:40.356927 31855 raft_consensus.cc:2229] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:47:40.356894 32050 tablet_service.cc:1514] Processing DeleteTablet for tablet 5368eebc10074fff8602cda978c81c9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:47:40 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:47968
I20260620 18:47:40.357005 31855 raft_consensus.cc:2258] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:40.357015 32146 tablet_replica.cc:330] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: stopping tablet replica
I20260620 18:47:40.357120 32146 raft_consensus.cc:2229] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:47:40.357161 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 31696
I20260620 18:47:40.357223 32146 raft_consensus.cc:2258] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:40.358654 32147 tablet_replica.cc:330] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: stopping tablet replica
I20260620 18:47:40.358775 32147 raft_consensus.cc:2229] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:47:40.358883 32147 raft_consensus.cc:2258] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:47:40.359356 32147 ts_tablet_manager.cc:1905] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:47:40.361367 32147 ts_tablet_manager.cc:1918] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:47:40.361447 32147 log.cc:1196] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/5368eebc10074fff8602cda978c81c9b
I20260620 18:47:40.361732 32147 ts_tablet_manager.cc:1939] T 5368eebc10074fff8602cda978c81c9b P dfeed14f4cf54e0391861a65e5c11b52: Deleting consensus metadata
I20260620 18:47:40.362172 31644 catalog_manager.cc:4841] TS dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685): tablet 5368eebc10074fff8602cda978c81c9b (table test-table [id=cbfe3190b36342faa1f2f3e94bbe88db]) successfully deleted
W20260620 18:47:40.363078 31643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:34631 (error 108)
I20260620 18:47:40.363698 32146 ts_tablet_manager.cc:1905] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 18:47:40.364151 32124 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:47:40.363501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34631 (local address 127.0.0.1:51556)
0620 18:47:40.363558 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:40.363578 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 18:47:40.363627 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:47:40.364110 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:34631: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 18:47:40.364558 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 31851
W20260620 18:47:40.364300 31643 catalog_manager.cc:4584] TS f0173537f098465b830c55067e2d2007 (127.15.77.65:34631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5368eebc10074fff8602cda978c81c9b: Network error: Client connection negotiation failed: client connection to 127.15.77.65:34631: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 18:47:40.370075 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 31987
I20260620 18:47:40.370194 32124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:47:40.370037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:41593 (local address 127.0.0.1:35152)
0620 18:47:40.370083 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 18:47:40.370129 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.66:41593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260620 18:47:40.374641 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 31625
I20260620 18:48:00.380041 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:37883
--webserver_interface=127.15.77.126
--webserver_port=43217
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:37883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:00.464418 32149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:00.464659 32149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:00.464726 32149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:00.465975 32149 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 18:48:00.466023 32149 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 18:48:00.466037 32149 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 18:48:00.466048 32149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:00.466063 32149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:00.466076 32149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:00.466089 32149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:00.466100 32149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:00.467694 32149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:37883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:37883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=43217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:00.467922 32149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:00.468117 32149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:00.473199 32155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:00.473434 32149 server_base.cc:1034] running on GCE node
W20260620 18:48:00.473199 32157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:00.473227 32154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:00.473778 32149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:00.474107 32149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:00.475306 32149 hybrid_clock.cc:648] HybridClock initialized: now 1781981280475292 us; error 48 us; skew 500 ppm
Jun 20 18:48:00 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981280, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:00.478132 32149 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:00.478641 32149 webserver.cc:460] Webserver started at http://127.15.77.126:43217/ using document root <none> and password file <none>
I20260620 18:48:00.478933 32149 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:00.478984 32149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:00.480561 32149 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:00.481371 32164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:00.481616 32149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:00.481705 32149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "e74d77882c5f4e5a8188a5829a9d8ccd"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "d3d9fc254cc3f951d6a055b7d63de6e7"
server_key_iv: "04866207aa8c3111f5c0f5efcd59590e"
server_key_version: "encryptionkey@0"
I20260620 18:48:00.482059 32149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:00.499490 32149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:00.501819 32149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:00.501940 32149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:00.506319 32149 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:37883
I20260620 18:48:00.506348 32216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:37883 every 8 connection(s)
I20260620 18:48:00.506783 32149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:00.508342 32217 sys_catalog.cc:263] Verifying existing consensus state
I20260620 18:48:00.509076 32217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Bootstrap starting.
I20260620 18:48:00.515643 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 32149
I20260620 18:48:00.515938 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:34631
--local_ip_for_outbound_sockets=127.15.77.65
--tserver_master_addrs=127.15.77.126:37883
--webserver_port=36121
--webserver_interface=127.15.77.65
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:48:00.517868 32217 log.cc:824] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:00.520344 32217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:48:00.520692 32217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Bootstrap complete.
I20260620 18:48:00.522956 32217 raft_consensus.cc:348] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:48:00.523208 32217 raft_consensus.cc:729] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74d77882c5f4e5a8188a5829a9d8ccd, State: Initialized, Role: FOLLOWER
I20260620 18:48:00.523435 32217 consensus_queue.cc:260] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:48:00.523525 32217 raft_consensus.cc:388] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:00.523581 32217 raft_consensus.cc:482] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:00.523628 32217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 2 FOLLOWER]: Advancing to term 3
I20260620 18:48:00.524614 32217 raft_consensus.cc:504] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:48:00.524739 32217 leader_election.cc:302] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e74d77882c5f4e5a8188a5829a9d8ccd; no voters:
I20260620 18:48:00.524958 32217 leader_election.cc:288] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 18:48:00.524992 32221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 3 FOLLOWER]: Leader election won for term 3
I20260620 18:48:00.525110 32221 raft_consensus.cc:686] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [term 3 LEADER]: Becoming Leader. State: Replica: e74d77882c5f4e5a8188a5829a9d8ccd, State: Running, Role: LEADER
I20260620 18:48:00.525241 32217 sys_catalog.cc:564] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:00.525259 32221 consensus_queue.cc:237] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } }
I20260620 18:48:00.525641 32222 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } } }
I20260620 18:48:00.525718 32222 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:00.526014 32223 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e74d77882c5f4e5a8188a5829a9d8ccd. Latest consensus state: current_term: 3 leader_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74d77882c5f4e5a8188a5829a9d8ccd" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 37883 } } }
I20260620 18:48:00.526090 32223 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:00.526216 32227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:00.527024 32227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:00.527127 32227 catalog_manager.cc:1259] Loaded cluster ID: 7f21d1fb44664cb483dca41b49fb68c0
I20260620 18:48:00.527206 32227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:00.528613 32227 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:00.528810 32227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Loaded TSK: 0
I20260620 18:48:00.528855 32227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e74d77882c5f4e5a8188a5829a9d8ccd: Loaded TSK: 1
F20260620 18:48:00.529111 32227 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4ca5d959ad google::LogMessage::Fail() at ??:0
@ 0x7f4ca5d9888a google::LogMessage::SendToLog() at ??:0
@ 0x7f4ca5d954de google::LogMessage::Flush() at ??:0
@ 0x7f4ca5d96a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4ca8502104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4ca84ffd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f4cadcef1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f4cadcef48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4cadcef594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f4cadcb5c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f4cadcbfad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4ca7c630d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4ca7c58954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4ca8abc6db start_thread at ??:0
@ 0x7f4ca489361f clone at ??:0
W20260620 18:48:00.638978 32219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:00.639272 32219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:00.639333 32219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:00.641206 32219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:00.641279 32219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:00.641325 32219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:48:00.643743 32219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:34631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=36121
--tserver_master_addrs=127.15.77.126:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:00.644053 32219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:00.644327 32219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:00.649971 32262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:00.649971 32259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:00.650243 32219 server_base.cc:1034] running on GCE node
W20260620 18:48:00.649984 32260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:00.650552 32219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:00.650825 32219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:00.652004 32219 hybrid_clock.cc:648] HybridClock initialized: now 1781981280651979 us; error 42 us; skew 500 ppm
Jun 20 18:48:00 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981280, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:00.654565 32219 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:48:00.655030 32219 webserver.cc:460] Webserver started at http://127.15.77.65:36121/ using document root <none> and password file <none>
I20260620 18:48:00.655279 32219 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:00.655375 32219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:00.656829 32219 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:00.657636 32269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:00.657981 32219 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:00.658092 32219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "f0173537f098465b830c55067e2d2007"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "f927108b301d3996500fc5f5d0fd5879"
server_key_iv: "26890c3aa80cc763da5978e5a7690405"
server_key_version: "encryptionkey@0"
I20260620 18:48:00.658442 32219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:00.668215 32219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:00.670745 32219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:00.670930 32219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:00.671191 32219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:00.671712 32276 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 18:48:00.673758 32219 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 18:48:00.673852 32219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 18:48:00.673911 32219 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 18:48:00.674638 32219 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 18:48:00.674706 32219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 18:48:00.674779 32276 tablet_bootstrap.cc:492] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: Bootstrap starting.
I20260620 18:48:00.682224 32219 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:34631
I20260620 18:48:00.682389 32383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:34631 every 8 connection(s)
I20260620 18:48:00.682683 32219 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:48:00.685868 32276 log.cc:824] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:00.689049 32276 tablet_bootstrap.cc:492] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 18:48:00.689384 32276 tablet_bootstrap.cc:492] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: Bootstrap complete.
I20260620 18:48:00.690623 32276 ts_tablet_manager.cc:1397] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260620 18:48:00.691799 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 32219
I20260620 18:48:00.692124 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:41593
--local_ip_for_outbound_sockets=127.15.77.66
--tserver_master_addrs=127.15.77.126:37883
--webserver_port=40623
--webserver_interface=127.15.77.66
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:48:00.692267 32276 raft_consensus.cc:348] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:48:00.692585 32276 raft_consensus.cc:729] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0173537f098465b830c55067e2d2007, State: Initialized, Role: FOLLOWER
I20260620 18:48:00.692752 32276 consensus_queue.cc:260] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:48:00.693046 32276 ts_tablet_manager.cc:1428] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007: Time spent starting tablet: real 0.002s user 0.003s sys 0.003s
W20260620 18:48:00.758431 32386 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:00.683561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:41273)
0620 18:48:00.683746 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:00.683792 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:00.684269 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:00.758170 (+ 73901us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
W20260620 18:48:00.758833 32384 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:37883 (0 consecutive failures): Network error: Failed to ping master at 127.15.77.126:37883: Client connection negotiation failed: client connection to 127.15.77.126:37883: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 18:48:00.759253 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:00.759126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:50871)
0620 18:48:00.759176 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:00.759218 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 18:48:00.759796 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:00.759688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:38661)
0620 18:48:00.759740 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:00.759766 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 18:48:00.759964 32384 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 18:48:00.780869 32389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:00.781095 32389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:00.781122 32389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:00.782349 32389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:00.782399 32389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:00.782428 32389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:48:00.784154 32389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:41593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=40623
--tserver_master_addrs=127.15.77.126:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:00.784441 32389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:00.784708 32389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:00.790375 32397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:00.790539 32395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:00.790406 32394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:00.790659 32389 server_base.cc:1034] running on GCE node
I20260620 18:48:00.790891 32389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:00.791116 32389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:00.792261 32389 hybrid_clock.cc:648] HybridClock initialized: now 1781981280792223 us; error 54 us; skew 500 ppm
Jun 20 18:48:00 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981280, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:00.794946 32389 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:48:00.795552 32389 webserver.cc:460] Webserver started at http://127.15.77.66:40623/ using document root <none> and password file <none>
I20260620 18:48:00.795809 32389 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:00.795857 32389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:00.797312 32389 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:48:00.798254 32404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:00.798483 32389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:00.798588 32389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "fe18e6f18f1b470fb672eb9eaa3945ca"
format_stamp: "Formatted at 2026-06-20 18:47:18 on dist-test-slave-tvl9"
server_key: "ac99d7169c3b788d8f7b67048eed8e40"
server_key_iv: "b6b940a6b85e09d9449bf0f70ac88688"
server_key_version: "encryptionkey@0"
I20260620 18:48:00.798945 32389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:00.810806 32389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:00.813431 32389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:00.813560 32389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:00.813762 32389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:00.814271 32411 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260620 18:48:00.815918 32411 ts_tablet_manager.cc:1852] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260620 18:48:00.816242 32411 ts_tablet_manager.cc:1905] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:00.818607 32411 ts_tablet_manager.cc:1918] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:00.818745 32411 log.cc:1196] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/5368eebc10074fff8602cda978c81c9b
I20260620 18:48:00.819051 32411 ts_tablet_manager.cc:1939] T 5368eebc10074fff8602cda978c81c9b P fe18e6f18f1b470fb672eb9eaa3945ca: Deleting consensus metadata
I20260620 18:48:00.819308 32389 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260620 18:48:00.819392 32389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260620 18:48:00.819423 32389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:00.819437 32389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:00.825520 32389 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:41593
I20260620 18:48:00.825673 32518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:41593 every 8 connection(s)
I20260620 18:48:00.825989 32389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:48:00.827142 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:00.826743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:35901)
0620 18:48:00.826923 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:00.827001 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
W20260620 18:48:00.827414 32519 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:37883 (0 consecutive failures): Network error: Failed to ping master at 127.15.77.126:37883: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
I20260620 18:48:00.827723 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:00.827626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:49735)
0620 18:48:00.827676 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:00.827704 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 18:48:00.827976 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:00.827926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:42185)
0620 18:48:00.827947 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:00.827965 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260620 18:48:00.828074 32519 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 18:48:00.828348 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 32389
I20260620 18:48:00.828577 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:37685
--local_ip_for_outbound_sockets=127.15.77.67
--tserver_master_addrs=127.15.77.126:37883
--webserver_port=41533
--webserver_interface=127.15.77.67
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:00.913539 32522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:00.913748 32522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:00.913774 32522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:00.915001 32522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:00.915050 32522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:00.915078 32522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:48:00.916707 32522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:37685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=41533
--tserver_master_addrs=127.15.77.126:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:00.916992 32522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:00.917254 32522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:00.922801 32530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:00.922811 32528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:00.922791 32527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:00.923015 32522 server_base.cc:1034] running on GCE node
I20260620 18:48:00.923261 32522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:00.923612 32522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:00.924767 32522 hybrid_clock.cc:648] HybridClock initialized: now 1781981280924752 us; error 29 us; skew 500 ppm
Jun 20 18:48:00 dist-test-slave-tvl9 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981280, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:00.927613 32522 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:48:00.928112 32522 webserver.cc:460] Webserver started at http://127.15.77.67:41533/ using document root <none> and password file <none>
I20260620 18:48:00.928368 32522 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:00.928414 32522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:00.929832 32522 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:00.930583 32537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:00.930773 32522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:00.930871 32522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "dfeed14f4cf54e0391861a65e5c11b52"
format_stamp: "Formatted at 2026-06-20 18:47:19 on dist-test-slave-tvl9"
server_key: "debfde5a65fbf57792eb726df7a724c2"
server_key_iv: "a32fce1ae4d467154f72e8e76d81047b"
server_key_version: "encryptionkey@0"
I20260620 18:48:00.931264 32522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:00.956902 32522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:00.959489 32522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:00.959622 32522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:00.959811 32522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:00.960175 32522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:00.960212 32522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:00.960235 32522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:00.960290 32522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:00.966392 32522 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:37685
I20260620 18:48:00.966504 32650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:37685 every 8 connection(s)
I20260620 18:48:00.966845 32522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:48:00.968092 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:00.967682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:49345)
0620 18:48:00.967869 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:00.967947 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":67,"threads_started":1}
W20260620 18:48:00.968577 32651 heartbeater.cc:644] Failed to heartbeat to 127.15.77.126:37883 (0 consecutive failures): Network error: Failed to ping master at 127.15.77.126:37883: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
I20260620 18:48:00.968925 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:00.968801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:49715)
0620 18:48:00.968868 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:00.968900 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 18:48:00.969249 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:00.969188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:48533)
0620 18:48:00.969210 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:00.969231 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260620 18:48:00.969388 32651 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 18:48:00.974233 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 32522
I20260620 18:48:01.760738 32655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:01.760384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:56025)
0620 18:48:01.760612 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:01.760674 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":70,"threads_started":1}
I20260620 18:48:01.828612 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:01.828398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:58517)
0620 18:48:01.828538 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:01.828574 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":55,"threads_started":1}
I20260620 18:48:01.970044 32657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:01.969774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:38451)
0620 18:48:01.969934 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:01.969993 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
I20260620 18:48:02.263478 32658 raft_consensus.cc:482] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:02.263613 32658 raft_consensus.cc:504] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:48:02.263947 32658 leader_election.cc:288] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593), dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685)
Jun 20 18:48:02 dist-test-slave-tvl9 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981280, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.66@KRBTEST.COM
Jun 20 18:48:02 dist-test-slave-tvl9 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981280, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.67@KRBTEST.COM
I20260620 18:48:02.280192 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:02.264268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:50707 (local address 127.15.77.66:41593)
0620 18:48:02.264442 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:02.264455 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:02.264519 (+ 64us) server_negotiation.cc:407] Connection header received
0620 18:48:02.265122 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:02.265124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:02.265164 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:02.265299 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:02.266319 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:02.267784 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:02.269388 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:02.270018 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:02.270138 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:02.276312 (+ 6174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:02.276316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:02.276316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:02.276328 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:02.278293 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:02.278768 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:02.278772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:02.278772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:02.278873 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:02.279172 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:02.279173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:02.279174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:02.279522 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:02.279622 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:02.279911 (+ 289us) server_negotiation.cc:299] Negotiation successful
0620 18:48:02.280052 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":60,"threads_started":1}
I20260620 18:48:02.280225 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:02.264120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:37685 (local address 127.15.77.65:51237)
0620 18:48:02.264269 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:02.264280 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:02.264335 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:02.265388 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:02.265391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:02.265896 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:02.266173 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:02.266181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:02.267917 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:02.267917 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:02.269282 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:02.269286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:02.270141 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:02.270143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:02.270309 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:02.274074 (+ 3765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:02.274086 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:02.276489 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:02.278636 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:02.278637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:02.278638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:02.278841 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:02.279165 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:02.279166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:02.279167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:02.279238 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:02.279726 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:02.279728 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:02.279783 (+ 55us) client_negotiation.cc:769] Sending connection context
0620 18:48:02.279884 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 18:48:02.280101 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":96,"thread_start_us":84,"threads_started":1}
I20260620 18:48:02.280316 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:02.264118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:41593 (local address 127.15.77.65:50707)
0620 18:48:02.264236 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:02.264249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:02.264312 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:02.265388 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:02.265391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:02.265896 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:02.266173 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:02.266181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:02.267914 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:02.267916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:02.269282 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:02.269286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:02.270143 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:02.270144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:02.270309 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:02.274074 (+ 3765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:02.274086 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:02.276149 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:02.278417 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:02.278426 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:02.278429 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:02.278665 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:02.278957 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:02.278959 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:02.278959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:02.279088 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:02.279599 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:02.279607 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:02.279745 (+ 138us) client_negotiation.cc:769] Sending connection context
0620 18:48:02.279858 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 18:48:02.280181 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":262,"thread_start_us":50,"threads_started":1}
I20260620 18:48:02.280579 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:02.264251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:51237 (local address 127.15.77.67:37685)
0620 18:48:02.264441 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:02.264455 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:02.264490 (+ 35us) server_negotiation.cc:407] Connection header received
0620 18:48:02.265078 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:02.265081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:02.265141 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:02.265299 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:02.266318 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:02.267784 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:02.269388 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:02.270020 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:02.270138 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:02.276643 (+ 6505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:02.276647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:02.276647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:02.276660 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:02.278485 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:02.278953 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:02.278956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:02.278956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:02.279072 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:02.279353 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:02.279354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:02.279355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:02.279631 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:02.279745 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:02.279971 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 18:48:02.280125 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":70,"threads_started":1}
W20260620 18:48:02.281122 32270 leader_election.cc:341] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593): Not found: Tablet not found: 5368eebc10074fff8602cda978c81c9b
W20260620 18:48:02.281416 32272 leader_election.cc:341] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685): Not found: Tablet not found: 5368eebc10074fff8602cda978c81c9b
I20260620 18:48:02.281469 32272 leader_election.cc:302] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0173537f098465b830c55067e2d2007; no voters: dfeed14f4cf54e0391861a65e5c11b52, fe18e6f18f1b470fb672eb9eaa3945ca
I20260620 18:48:02.281580 32658 raft_consensus.cc:2726] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:48:02.761554 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:02.761367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:56827)
0620 18:48:02.761460 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:02.761511 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260620 18:48:02.829398 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:02.829147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:37027)
0620 18:48:02.829311 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:02.829354 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":64,"threads_started":1}
I20260620 18:48:02.970841 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:02.970611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:45267)
0620 18:48:02.970744 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:02.970792 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260620 18:48:03.762269 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:03.762080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:42743)
0620 18:48:03.762184 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:03.762220 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":59,"threads_started":1}
I20260620 18:48:03.830253 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:03.830013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:56023)
0620 18:48:03.830159 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:03.830206 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260620 18:48:03.971699 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:03.971473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:45779)
0620 18:48:03.971604 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:03.971651 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260620 18:48:04.022451 32668 raft_consensus.cc:482] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:04.022562 32668 raft_consensus.cc:504] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:48:04.022742 32668 leader_election.cc:288] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593), dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685)
W20260620 18:48:04.023207 32272 leader_election.cc:341] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685): Not found: Tablet not found: 5368eebc10074fff8602cda978c81c9b
W20260620 18:48:04.023296 32270 leader_election.cc:341] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593): Not found: Tablet not found: 5368eebc10074fff8602cda978c81c9b
I20260620 18:48:04.023420 32270 leader_election.cc:302] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0173537f098465b830c55067e2d2007; no voters: dfeed14f4cf54e0391861a65e5c11b52, fe18e6f18f1b470fb672eb9eaa3945ca
I20260620 18:48:04.023527 32668 raft_consensus.cc:2726] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:48:04.763091 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:04.762851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:54403)
0620 18:48:04.762994 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:04.763043 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260620 18:48:04.831207 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:04.830931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:38077)
0620 18:48:04.831108 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:04.831160 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":72,"threads_started":1}
I20260620 18:48:04.972590 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:04.972289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:34067)
0620 18:48:04.972474 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:04.972523 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":81,"threads_started":1}
I20260620 18:48:05.763859 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:05.763635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:48481)
0620 18:48:05.763763 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:05.763809 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260620 18:48:05.831974 32673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:05.831754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:34777)
0620 18:48:05.831890 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:05.831931 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260620 18:48:05.944141 32674 raft_consensus.cc:482] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:05.944257 32674 raft_consensus.cc:504] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:48:05.944505 32674 leader_election.cc:288] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593), dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685)
W20260620 18:48:05.944957 32270 leader_election.cc:341] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593): Not found: Tablet not found: 5368eebc10074fff8602cda978c81c9b
W20260620 18:48:05.945056 32272 leader_election.cc:341] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685): Not found: Tablet not found: 5368eebc10074fff8602cda978c81c9b
I20260620 18:48:05.945122 32272 leader_election.cc:302] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0173537f098465b830c55067e2d2007; no voters: dfeed14f4cf54e0391861a65e5c11b52, fe18e6f18f1b470fb672eb9eaa3945ca
I20260620 18:48:05.945230 32674 raft_consensus.cc:2726] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:48:05.973359 32675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:05.973134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:51119)
0620 18:48:05.973269 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:05.973313 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260620 18:48:06.764643 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:06.764380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:59119)
0620 18:48:06.764534 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:06.764592 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260620 18:48:06.832864 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:06.832545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:53239)
0620 18:48:06.832740 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:06.832806 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":134,"threads_started":1}
I20260620 18:48:06.974154 32678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:06.973891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:44725)
0620 18:48:06.974059 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:06.974105 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":74,"threads_started":1}
I20260620 18:48:07.765395 32679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:07.765168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:35103)
0620 18:48:07.765305 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:07.765348 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260620 18:48:07.833695 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:07.833464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:46371)
0620 18:48:07.833604 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:07.833648 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260620 18:48:07.974874 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:07.974661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:50181)
0620 18:48:07.974789 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:07.974825 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260620 18:48:08.452867 32682 raft_consensus.cc:482] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:08.452962 32682 raft_consensus.cc:504] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:48:08.453171 32682 leader_election.cc:288] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593), dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685)
W20260620 18:48:08.453588 32270 leader_election.cc:341] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593): Not found: Tablet not found: 5368eebc10074fff8602cda978c81c9b
W20260620 18:48:08.453680 32272 leader_election.cc:341] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685): Not found: Tablet not found: 5368eebc10074fff8602cda978c81c9b
I20260620 18:48:08.453744 32272 leader_election.cc:302] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0173537f098465b830c55067e2d2007; no voters: dfeed14f4cf54e0391861a65e5c11b52, fe18e6f18f1b470fb672eb9eaa3945ca
I20260620 18:48:08.453837 32682 raft_consensus.cc:2726] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:48:08.766320 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:08.766033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:43989)
0620 18:48:08.766189 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:08.766258 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260620 18:48:08.834445 32684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:08.834188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:51479)
0620 18:48:08.834358 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:08.834402 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
I20260620 18:48:08.975731 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:08.975510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:50145)
0620 18:48:08.975636 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:08.975681 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260620 18:48:09.767115 32686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:09.766884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:37939)
0620 18:48:09.767018 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:09.767064 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260620 18:48:09.835134 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:09.834883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:48851)
0620 18:48:09.835043 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:09.835076 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260620 18:48:09.976452 32688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:09.976226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:58617)
0620 18:48:09.976366 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:09.976405 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260620 18:48:10.767957 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:10.767720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:45139)
0620 18:48:10.767866 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:10.767908 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260620 18:48:10.835843 32690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:10.835592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:33775)
0620 18:48:10.835749 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:10.835795 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
I20260620 18:48:10.977272 32691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:10.977002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:41005)
0620 18:48:10.977146 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:10.977212 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":109,"threads_started":1}
I20260620 18:48:11.131770 32692 raft_consensus.cc:482] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:11.131865 32692 raft_consensus.cc:504] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe18e6f18f1b470fb672eb9eaa3945ca" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 41593 } } peers { permanent_uuid: "dfeed14f4cf54e0391861a65e5c11b52" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 37685 } } peers { permanent_uuid: "f0173537f098465b830c55067e2d2007" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34631 } }
I20260620 18:48:11.132117 32692 leader_election.cc:288] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593), dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685)
W20260620 18:48:11.132498 32272 leader_election.cc:341] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dfeed14f4cf54e0391861a65e5c11b52 (127.15.77.67:37685): Not found: Tablet not found: 5368eebc10074fff8602cda978c81c9b
W20260620 18:48:11.132596 32270 leader_election.cc:341] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe18e6f18f1b470fb672eb9eaa3945ca (127.15.77.66:41593): Not found: Tablet not found: 5368eebc10074fff8602cda978c81c9b
I20260620 18:48:11.132673 32270 leader_election.cc:302] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0173537f098465b830c55067e2d2007; no voters: dfeed14f4cf54e0391861a65e5c11b52, fe18e6f18f1b470fb672eb9eaa3945ca
I20260620 18:48:11.132740 32692 raft_consensus.cc:2726] T 5368eebc10074fff8602cda978c81c9b P f0173537f098465b830c55067e2d2007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 18:48:11.768695 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:11.768458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:55573)
0620 18:48:11.768597 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:11.768645 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260620 18:48:11.836540 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:11.836334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:37821)
0620 18:48:11.836455 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:11.836495 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260620 18:48:11.977963 32695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:11.977720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:53737)
0620 18:48:11.977868 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:11.977914 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":61,"threads_started":1}
I20260620 18:48:12.769551 32696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:12.769304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:45701)
0620 18:48:12.769455 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:12.769503 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260620 18:48:12.837294 32697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:12.837059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:53793)
0620 18:48:12.837203 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:12.837249 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260620 18:48:12.978714 32698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:12.978469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:51633)
0620 18:48:12.978598 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:12.978653 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260620 18:48:13.770380 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:13.770129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:45421)
0620 18:48:13.770285 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:13.770332 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":61,"threads_started":1}
I20260620 18:48:13.838057 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:13.837840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:51811)
0620 18:48:13.837979 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:13.838012 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260620 18:48:13.979534 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:13.979236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:60493)
0620 18:48:13.979431 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:13.979469 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":59,"threads_started":1}
I20260620 18:48:14.771152 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:14.770911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:49105)
0620 18:48:14.771058 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:14.771103 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260620 18:48:14.838824 32703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:14.838581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:39541)
0620 18:48:14.838720 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:14.838769 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260620 18:48:14.980264 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:14.980019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:43333)
0620 18:48:14.980176 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:14.980215 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260620 18:48:15.772043 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:15.771766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.65:34293)
0620 18:48:15.771929 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:15.771997 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":66,"threads_started":1}
I20260620 18:48:15.839720 32706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:15.839489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.66:46753)
0620 18:48:15.839633 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:15.839672 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260620 18:48:15.981046 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:15.980794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:37883 (local address 127.15.77.67:42053)
0620 18:48:15.980938 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:15.980998 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.126:37883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260620 18:48:15.983677 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 32219
I20260620 18:48:15.988867 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 32389
I20260620 18:48:15.993330 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 32522
2026-06-20T18:48:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58740 ms)
[----------] 30 tests from SecurityITest (263087 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:16 dist-test-slave-tvl9 krb5kdc[32710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:16 dist-test-slave-tvl9 krb5kdc[32710](info): set up 2 sockets
Jun 20 18:48:16 dist-test-slave-tvl9 krb5kdc[32710](info): commencing operation
krb5kdc: starting...
W20260620 18:48:17.055042 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:17 dist-test-slave-tvl9 krb5kdc[32710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:17.174721 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:36221
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:36221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:17.259692 32723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:17.259927 32723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:17.259967 32723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:17.261157 32723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:17.261226 32723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:17.261251 32723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:17.261273 32723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:17.261301 32723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:17.262811 32723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:36221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:36221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:17.263103 32723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:17.263375 32723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:17.268347 32729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:17.268332 32728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:17.268327 32731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:17.268359 32723 server_base.cc:1034] running on GCE node
I20260620 18:48:17.268786 32723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:17.269150 32723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:17.270313 32723 hybrid_clock.cc:648] HybridClock initialized: now 1781981297270291 us; error 36 us; skew 500 ppm
Jun 20 18:48:17 dist-test-slave-tvl9 krb5kdc[32710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981297, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:17.272856 32723 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:17.273355 32723 webserver.cc:460] Webserver started at http://127.15.77.126:46099/ using document root <none> and password file <none>
I20260620 18:48:17.273584 32723 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:17.273650 32723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:17.273804 32723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:17.274895 32723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "c92fdbf5de884d1aa5c5896faad7f706"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "4143adc5bf94e3286307d5d2120526ec"
server_key_iv: "fe220cf14a5359a881ef049465047854"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.275259 32723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "c92fdbf5de884d1aa5c5896faad7f706"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "4143adc5bf94e3286307d5d2120526ec"
server_key_iv: "fe220cf14a5359a881ef049465047854"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.276644 32723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:17.277401 32738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:17.277601 32723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:17.277702 32723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "c92fdbf5de884d1aa5c5896faad7f706"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "4143adc5bf94e3286307d5d2120526ec"
server_key_iv: "fe220cf14a5359a881ef049465047854"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.277784 32723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:17.292104 32723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:17.294145 32723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:17.294315 32723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:17.299006 32723 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:36221
I20260620 18:48:17.299045 322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:36221 every 8 connection(s)
I20260620 18:48:17.299504 32723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:17.299846 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 32723
I20260620 18:48:17.299959 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:17.300161 15669 external_mini_cluster.cc:1428] Setting key 6b6987ef95bec902492dfff8382f0cc6
I20260620 18:48:17.300689 323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:17.304155 323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706: Bootstrap starting.
I20260620 18:48:17.304831 323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:17.305171 323 log.cc:824] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:17.305989 323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706: No bootstrap required, opened a new log
Jun 20 18:48:17 dist-test-slave-tvl9 krb5kdc[32710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:17.308355 323 raft_consensus.cc:348] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fdbf5de884d1aa5c5896faad7f706" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36221 } }
I20260620 18:48:17.308513 323 raft_consensus.cc:374] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:17.308537 323 raft_consensus.cc:729] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c92fdbf5de884d1aa5c5896faad7f706, State: Initialized, Role: FOLLOWER
I20260620 18:48:17.308684 323 consensus_queue.cc:260] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fdbf5de884d1aa5c5896faad7f706" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36221 } }
I20260620 18:48:17.308779 323 raft_consensus.cc:388] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:17.308823 323 raft_consensus.cc:482] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:17.308892 323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:17.309471 323 raft_consensus.cc:504] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fdbf5de884d1aa5c5896faad7f706" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36221 } }
I20260620 18:48:17.309590 323 leader_election.cc:302] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c92fdbf5de884d1aa5c5896faad7f706; no voters:
I20260620 18:48:17.309756 323 leader_election.cc:288] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:17.309829 328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:17.310053 323 sys_catalog.cc:564] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:17.310066 328 raft_consensus.cc:686] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [term 1 LEADER]: Becoming Leader. State: Replica: c92fdbf5de884d1aa5c5896faad7f706, State: Running, Role: LEADER
I20260620 18:48:17.310408 328 consensus_queue.cc:237] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fdbf5de884d1aa5c5896faad7f706" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36221 } }
I20260620 18:48:17.310818 330 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c92fdbf5de884d1aa5c5896faad7f706. Latest consensus state: current_term: 1 leader_uuid: "c92fdbf5de884d1aa5c5896faad7f706" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fdbf5de884d1aa5c5896faad7f706" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36221 } } }
I20260620 18:48:17.310984 330 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:17.311684 329 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c92fdbf5de884d1aa5c5896faad7f706" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92fdbf5de884d1aa5c5896faad7f706" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36221 } } }
I20260620 18:48:17.312058 329 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706 [sys.catalog]: This master's current role is: LEADER
W20260620 18:48:17.312081 344 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:17.312191 344 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:17.312265 337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:17.312503 337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:17.312564 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.300770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.15.77.126:36221)
0620 18:48:17.300959 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.300969 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.301007 (+ 38us) server_negotiation.cc:407] Connection header received
0620 18:48:17.301655 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.301665 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.301739 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.301888 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:17.302492 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.303889 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.304815 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.305444 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.305549 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.308370 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:17.308374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:17.308375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:17.308387 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:17.310581 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:17.311033 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:17.311036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:17.311036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:17.311136 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:17.311396 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:17.311397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:17.311397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:17.311724 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:17.311895 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.312191 (+ 296us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.312338 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260620 18:48:17.314074 337 catalog_manager.cc:1347] Generated new cluster ID: 5c50849584ed414196212e99814d98a8
I20260620 18:48:17.314131 337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:17.337829 337 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:17.338403 337 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:17.341004 337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706: Generated new TSK 0
I20260620 18:48:17.341203 337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:48:17.377946 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:36221
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:17.476658 349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:17.476866 349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:17.476889 349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:17.477996 349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:17.478031 349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:17.478057 349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:48:17.479491 349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:36221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:17.479686 349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:17.479879 349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:17.485208 354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:17.485208 357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:17.485450 349 server_base.cc:1034] running on GCE node
W20260620 18:48:17.485370 355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:17.485716 349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:17.485919 349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:17.487071 349 hybrid_clock.cc:648] HybridClock initialized: now 1781981297487039 us; error 58 us; skew 500 ppm
Jun 20 18:48:17 dist-test-slave-tvl9 krb5kdc[32710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981297, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:17.489728 349 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:48:17.490162 349 webserver.cc:460] Webserver started at http://127.15.77.65:45419/ using document root <none> and password file <none>
I20260620 18:48:17.490402 349 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:17.490453 349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:17.490623 349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:17.491743 349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "b4c2092e39bf4245aa030043f3e35a53"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "206597287c0052939d1e1e5248d71ebc"
server_key_iv: "2fdaaf428fcf948b3cee7e5bf1f1f3b2"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.492089 349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "b4c2092e39bf4245aa030043f3e35a53"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "206597287c0052939d1e1e5248d71ebc"
server_key_iv: "2fdaaf428fcf948b3cee7e5bf1f1f3b2"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.493353 349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:17.494078 364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:17.494267 349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:17.494338 349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "b4c2092e39bf4245aa030043f3e35a53"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "206597287c0052939d1e1e5248d71ebc"
server_key_iv: "2fdaaf428fcf948b3cee7e5bf1f1f3b2"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.494429 349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:17.521405 349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:17.523934 349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:17.524113 349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:17.524360 349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:17.524703 349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:17.524737 349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:17.524782 349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:17.524809 349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:17.530691 349 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:33141
I20260620 18:48:17.530725 477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:33141 every 8 connection(s)
I20260620 18:48:17.531059 349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:48:17.534097 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 349
I20260620 18:48:17.534235 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:48:17.534425 15669 external_mini_cluster.cc:1428] Setting key 0a4fbd02562a78b9b734347862fd3496
Jun 20 18:48:17 dist-test-slave-tvl9 krb5kdc[32710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981297, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:17.541247 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.531788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:45595 (local address 127.15.77.126:36221)
0620 18:48:17.531832 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.531834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.532043 (+ 209us) server_negotiation.cc:407] Connection header received
0620 18:48:17.532565 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.532566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.532577 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.532642 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:17.533626 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.534129 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.535530 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.535741 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.535822 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.538157 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:17.538159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:17.538159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:17.538168 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:17.539794 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:17.540240 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:17.540241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:17.540242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:17.540301 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:17.540566 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:17.540567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:17.540567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:17.540697 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:17.540769 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.541069 (+ 300us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.541154 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:48:17.541447 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.531759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36221 (local address 127.15.77.65:45595)
0620 18:48:17.531929 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:17.531972 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:17.532453 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:17.532726 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:17.532730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:17.533301 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:17.533519 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.533525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.534257 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.534258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.535433 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.535436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.535831 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.535833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.535980 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.536527 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:17.536535 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:17.538053 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:17.539898 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:17.539906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:17.539908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:17.540144 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:17.540377 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:17.540378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:17.540378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:17.540494 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:17.540771 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:17.540780 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:17.540922 (+ 142us) client_negotiation.cc:769] Sending connection context
0620 18:48:17.541036 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 18:48:17.541200 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260620 18:48:17.541994 478 heartbeater.cc:347] Connected to a master server at 127.15.77.126:36221
I20260620 18:48:17.542093 478 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:17.542295 478 heartbeater.cc:511] Master 127.15.77.126:36221 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:48:17.542876 32754 ts_manager.cc:194] Registered new tserver with Master: b4c2092e39bf4245aa030043f3e35a53 (127.15.77.65:33141)
I20260620 18:48:17.543565 32754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:45595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:48:17.559208 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:36221
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:17.640607 483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:17.640879 483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:17.640921 483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:17.642115 483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:17.642191 483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:17.642237 483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:48:17.643754 483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:36221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:17.644055 483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:17.644304 483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:17.649413 491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:17.649439 489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:17.649500 488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:17.649621 483 server_base.cc:1034] running on GCE node
I20260620 18:48:17.649930 483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:17.650187 483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:17.651377 483 hybrid_clock.cc:648] HybridClock initialized: now 1781981297651355 us; error 30 us; skew 500 ppm
Jun 20 18:48:17 dist-test-slave-tvl9 krb5kdc[32710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981297, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:17.653843 483 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:48:17.654285 483 webserver.cc:460] Webserver started at http://127.15.77.66:35961/ using document root <none> and password file <none>
I20260620 18:48:17.654515 483 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:17.654558 483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:17.654651 483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:17.655714 483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "ee66d316287047b08f721ffe2b91af8f"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "05b04ef0a88c265b03192dbcad0ced73"
server_key_iv: "5a70303d5aacbfa550e98c80131a62a0"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.656015 483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "ee66d316287047b08f721ffe2b91af8f"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "05b04ef0a88c265b03192dbcad0ced73"
server_key_iv: "5a70303d5aacbfa550e98c80131a62a0"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.657243 483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:48:17.657994 498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:17.658185 483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:17.658294 483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "ee66d316287047b08f721ffe2b91af8f"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "05b04ef0a88c265b03192dbcad0ced73"
server_key_iv: "5a70303d5aacbfa550e98c80131a62a0"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.658370 483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:17.672205 483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:17.674154 483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:17.674315 483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:17.674561 483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:17.674865 483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:17.674917 483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:17.674962 483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:17.674988 483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:17.681046 483 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:36389
I20260620 18:48:17.681121 611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:36389 every 8 connection(s)
I20260620 18:48:17.681452 483 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:48:17.683925 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 483
I20260620 18:48:17.684033 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:48:17.684196 15669 external_mini_cluster.cc:1428] Setting key 2f9a64da82a60c71293307968726c759
Jun 20 18:48:17 dist-test-slave-tvl9 krb5kdc[32710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981297, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:17.691056 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.682167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:52821 (local address 127.15.77.126:36221)
0620 18:48:17.682231 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.682233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.682462 (+ 229us) server_negotiation.cc:407] Connection header received
0620 18:48:17.682939 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.682940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.682950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.683018 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:17.684041 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.684628 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.685662 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.685878 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.685957 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.688177 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:17.688179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:17.688180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:17.688187 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:17.689611 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:17.690046 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:17.690047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:17.690048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:17.690112 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:17.690372 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:17.690373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:17.690373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:17.690514 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:17.690587 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.690866 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.690954 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 18:48:17.691268 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.682165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36221 (local address 127.15.77.66:52821)
0620 18:48:17.682329 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:17.682372 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:17.682858 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:17.683072 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:17.683076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:17.683720 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:17.683930 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.683941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.684727 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.684728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.685567 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.685569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.685968 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.685969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.686107 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.686652 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:17.686660 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:17.688078 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:17.689712 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:17.689720 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:17.689722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:17.689967 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:17.690179 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:17.690180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:17.690180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:17.690301 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:17.690587 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:17.690595 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:17.690735 (+ 140us) client_negotiation.cc:769] Sending connection context
0620 18:48:17.690845 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 18:48:17.691027 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260620 18:48:17.691888 612 heartbeater.cc:347] Connected to a master server at 127.15.77.126:36221
I20260620 18:48:17.692000 612 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:17.692200 612 heartbeater.cc:511] Master 127.15.77.126:36221 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
I20260620 18:48:17.692629 32754 ts_manager.cc:194] Registered new tserver with Master: ee66d316287047b08f721ffe2b91af8f (127.15.77.66:36389)
I20260620 18:48:17.693017 32754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:52821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:48:17.710952 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:36221
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:17.793318 617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:17.793515 617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:17.793538 617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:17.794618 617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:17.794653 617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:17.794679 617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:48:17.796177 617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:36221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:17.796367 617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:17.796603 617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:17.801748 622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:17.801748 623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:17.801743 625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:17.802093 617 server_base.cc:1034] running on GCE node
I20260620 18:48:17.802364 617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:17.802625 617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:17.803794 617 hybrid_clock.cc:648] HybridClock initialized: now 1781981297803783 us; error 40 us; skew 500 ppm
Jun 20 18:48:17 dist-test-slave-tvl9 krb5kdc[32710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981297, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:17.806282 617 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:48:17.806723 617 webserver.cc:460] Webserver started at http://127.15.77.67:34061/ using document root <none> and password file <none>
I20260620 18:48:17.806964 617 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:17.807035 617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:17.807178 617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:17.808267 617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "492da6fe2729466797e4184e0a7b3a49"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "c74ce212db3890ffbbb63313e7d06063"
server_key_iv: "43b8d860cfd42087fffe34cfba8a2799"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.808619 617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "492da6fe2729466797e4184e0a7b3a49"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "c74ce212db3890ffbbb63313e7d06063"
server_key_iv: "43b8d860cfd42087fffe34cfba8a2799"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.809943 617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:48:17.810688 632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:17.810881 617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:17.811022 617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "492da6fe2729466797e4184e0a7b3a49"
format_stamp: "Formatted at 2026-06-20 18:48:17 on dist-test-slave-tvl9"
server_key: "c74ce212db3890ffbbb63313e7d06063"
server_key_iv: "43b8d860cfd42087fffe34cfba8a2799"
server_key_version: "encryptionkey@0"
I20260620 18:48:17.811100 617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:17.828864 617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:17.830904 617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:17.831073 617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:17.831357 617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:17.831712 617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:17.831768 617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:17.831812 617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:17.831838 617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:17.837519 617 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:45761
I20260620 18:48:17.837592 745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:45761 every 8 connection(s)
I20260620 18:48:17.837929 617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
Jun 20 18:48:17 dist-test-slave-tvl9 krb5kdc[32710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981297, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:17.846208 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 617
I20260620 18:48:17.846315 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:48:17.846513 15669 external_mini_cluster.cc:1428] Setting key ed66c838f112bad5919c1939cdfa4a49
I20260620 18:48:17.848148 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.838730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:45747 (local address 127.15.77.126:36221)
0620 18:48:17.838806 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.838807 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.839021 (+ 214us) server_negotiation.cc:407] Connection header received
0620 18:48:17.839615 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.839617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.839628 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.839689 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:17.840752 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.841446 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.842513 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.842711 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.842770 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.845000 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:17.845003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:17.845003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:17.845010 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:17.846655 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:17.847186 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:17.847186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:17.847187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:17.847251 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:17.847579 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:17.847580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:17.847580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:17.847702 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:17.847759 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.848021 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.848085 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 18:48:17.848341 748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.838632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36221 (local address 127.15.77.67:45747)
0620 18:48:17.838877 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:17.838915 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:17.839498 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:17.839815 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:17.839819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:17.840408 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:17.840622 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.840627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.841554 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.841555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.842378 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.842380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.842789 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.842790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.842899 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.843517 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:17.843525 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:17.844913 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:17.846738 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:17.846745 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:17.846748 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:17.847118 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:17.847338 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:17.847339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:17.847339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:17.847480 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:17.847760 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:17.847765 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:17.847884 (+ 119us) client_negotiation.cc:769] Sending connection context
0620 18:48:17.847979 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 18:48:17.848101 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":103,"threads_started":1}
I20260620 18:48:17.848848 746 heartbeater.cc:347] Connected to a master server at 127.15.77.126:36221
I20260620 18:48:17.848929 746 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:17.849072 746 heartbeater.cc:511] Master 127.15.77.126:36221 requested a full tablet report, sending...
I20260620 18:48:17.849383 32754 ts_manager.cc:194] Registered new tserver with Master: 492da6fe2729466797e4184e0a7b3a49 (127.15.77.67:45761)
I20260620 18:48:17.849737 32754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:45747
I20260620 18:48:17.858363 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:48:17.865933 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.859813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44866 (local address 127.15.77.126:36221)
0620 18:48:17.859874 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.859876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.860016 (+ 140us) server_negotiation.cc:407] Connection header received
0620 18:48:17.860059 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.860059 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.860068 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.860123 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:17.860695 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.861193 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.861980 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.862157 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.862204 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.863452 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:17.863453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:17.863453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:17.863459 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:17.864888 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:17.865237 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:17.865238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:17.865239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:17.865290 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:17.865469 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:17.865469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:17.865469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:17.865637 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:17.865714 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.865809 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.865868 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 18:48:17.867792 32754 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:48:17.869372 32754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:48:17.876171 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.871812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47066 (local address 127.15.77.66:36389)
0620 18:48:17.872176 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.872179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.872191 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:48:17.872238 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.872240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.872257 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.872337 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:17.872725 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.873556 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.874909 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.875651 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.875720 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.875775 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.876044 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.876087 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":103,"threads_started":1}
I20260620 18:48:17.876164 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.871712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:45761 (local address 127.0.0.1:45506)
0620 18:48:17.872043 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:17.872087 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:17.872152 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:17.872466 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:17.872469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:17.872510 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:17.872622 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.872625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.873937 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.873938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.875008 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.875011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.875818 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.875820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.875902 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.875909 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 18:48:17.876023 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 18:48:17.876069 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":55,"thread_start_us":78,"threads_started":1}
I20260620 18:48:17.876166 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.871708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:36389 (local address 127.0.0.1:47066)
0620 18:48:17.872043 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:17.872087 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:17.872152 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:17.872542 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:17.872543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:17.872546 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:17.872652 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.872654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.873655 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.873656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.874814 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.874817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.875811 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.875812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.875896 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.875909 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 18:48:17.875999 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 18:48:17.876059 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":32,"thread_start_us":59,"threads_started":1}
I20260620 18:48:17.876334 761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.871935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33141 (local address 127.0.0.1:59126)
0620 18:48:17.872231 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:17.872262 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:17.872316 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:17.872931 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:17.872932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:17.872934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:17.872976 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.872978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.874143 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.874144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.875175 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.875177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.876123 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.876124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.876185 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.876188 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:48:17.876237 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 18:48:17.876276 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":55,"threads_started":1}
I20260620 18:48:17.876526 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.871810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45506 (local address 127.15.77.67:45761)
0620 18:48:17.872088 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.872094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.872239 (+ 145us) server_negotiation.cc:407] Connection header received
0620 18:48:17.872289 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.872292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.872308 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.872377 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:17.872710 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.873823 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.875094 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.875721 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.875807 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.875863 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.876395 (+ 532us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.876453 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":112,"threads_started":1}
I20260620 18:48:17.876596 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.872058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.15.77.65:33141)
0620 18:48:17.872770 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.872776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.872785 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:17.872822 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.872824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.872838 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.872918 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:17.873043 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.874053 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.875252 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.875981 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.876040 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.876089 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.876452 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.876508 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":83,"threads_started":1}
I20260620 18:48:17.877189 412 tablet_service.cc:1467] Processing CreateTablet for tablet 53d05e0980e040d8a4f77eca887da665 (DEFAULT_TABLE table=test-table [id=a0a08cb7c08943b58a6eb04add8e5916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:17.877189 680 tablet_service.cc:1467] Processing CreateTablet for tablet 53d05e0980e040d8a4f77eca887da665 (DEFAULT_TABLE table=test-table [id=a0a08cb7c08943b58a6eb04add8e5916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:17.877559 412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d05e0980e040d8a4f77eca887da665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:17.877559 680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d05e0980e040d8a4f77eca887da665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:17.879990 546 tablet_service.cc:1467] Processing CreateTablet for tablet 53d05e0980e040d8a4f77eca887da665 (DEFAULT_TABLE table=test-table [id=a0a08cb7c08943b58a6eb04add8e5916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:17.880311 546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d05e0980e040d8a4f77eca887da665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:17.881052 763 tablet_bootstrap.cc:492] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: Bootstrap starting.
I20260620 18:48:17.881770 763 tablet_bootstrap.cc:654] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:17.882362 763 log.cc:824] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:17.882712 764 tablet_bootstrap.cc:492] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53: Bootstrap starting.
I20260620 18:48:17.883157 763 tablet_bootstrap.cc:492] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: No bootstrap required, opened a new log
I20260620 18:48:17.883301 763 ts_tablet_manager.cc:1397] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:48:17.883525 764 tablet_bootstrap.cc:654] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:17.883771 766 tablet_bootstrap.cc:492] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: Bootstrap starting.
I20260620 18:48:17.884115 764 log.cc:824] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:17.884450 766 tablet_bootstrap.cc:654] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:17.884822 764 tablet_bootstrap.cc:492] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53: No bootstrap required, opened a new log
I20260620 18:48:17.884905 764 ts_tablet_manager.cc:1397] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:48:17.885025 766 log.cc:824] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:17.885505 763 raft_consensus.cc:348] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 } } peers { permanent_uuid: "492da6fe2729466797e4184e0a7b3a49" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45761 } } peers { permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 } }
I20260620 18:48:17.885690 763 raft_consensus.cc:374] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:17.885732 763 raft_consensus.cc:729] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492da6fe2729466797e4184e0a7b3a49, State: Initialized, Role: FOLLOWER
I20260620 18:48:17.885780 766 tablet_bootstrap.cc:492] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: No bootstrap required, opened a new log
I20260620 18:48:17.885857 766 ts_tablet_manager.cc:1397] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:48:17.885902 763 consensus_queue.cc:260] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 } } peers { permanent_uuid: "492da6fe2729466797e4184e0a7b3a49" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45761 } } peers { permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 } }
I20260620 18:48:17.886219 746 heartbeater.cc:503] Master 127.15.77.126:36221 was elected leader, sending a full tablet report...
I20260620 18:48:17.886225 763 ts_tablet_manager.cc:1428] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:48:17.887229 764 raft_consensus.cc:348] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 } } peers { permanent_uuid: "492da6fe2729466797e4184e0a7b3a49" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45761 } } peers { permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 } }
I20260620 18:48:17.887492 764 raft_consensus.cc:374] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:17.887589 764 raft_consensus.cc:729] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c2092e39bf4245aa030043f3e35a53, State: Initialized, Role: FOLLOWER
I20260620 18:48:17.887729 764 consensus_queue.cc:260] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 } } peers { permanent_uuid: "492da6fe2729466797e4184e0a7b3a49" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45761 } } peers { permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 } }
I20260620 18:48:17.887763 766 raft_consensus.cc:348] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 } } peers { permanent_uuid: "492da6fe2729466797e4184e0a7b3a49" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45761 } } peers { permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 } }
I20260620 18:48:17.887955 766 raft_consensus.cc:374] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:17.888032 764 ts_tablet_manager.cc:1428] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 18:48:17.888083 766 raft_consensus.cc:729] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee66d316287047b08f721ffe2b91af8f, State: Initialized, Role: FOLLOWER
I20260620 18:48:17.888031 478 heartbeater.cc:503] Master 127.15.77.126:36221 was elected leader, sending a full tablet report...
I20260620 18:48:17.888198 766 consensus_queue.cc:260] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 } } peers { permanent_uuid: "492da6fe2729466797e4184e0a7b3a49" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45761 } } peers { permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 } }
I20260620 18:48:17.888474 766 ts_tablet_manager.cc:1428] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260620 18:48:17.888536 612 heartbeater.cc:503] Master 127.15.77.126:36221 was elected leader, sending a full tablet report...
I20260620 18:48:17.922387 769 raft_consensus.cc:482] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:17.922549 769 raft_consensus.cc:504] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 } } peers { permanent_uuid: "492da6fe2729466797e4184e0a7b3a49" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45761 } } peers { permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 } }
I20260620 18:48:17.922873 769 leader_election.cc:288] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee66d316287047b08f721ffe2b91af8f (127.15.77.66:36389), b4c2092e39bf4245aa030043f3e35a53 (127.15.77.65:33141)
I20260620 18:48:17.925694 748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.923052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33141 (local address 127.15.77.67:49369)
0620 18:48:17.923098 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:17.923113 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:17.923173 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:17.923393 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:17.923395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:17.923397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:17.923477 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.923478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.924119 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.924120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.924850 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.924852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.925420 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.925421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.925503 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.925506 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:48:17.925573 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 18:48:17.925617 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":1}
I20260620 18:48:17.925694 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.923191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:49369 (local address 127.15.77.65:33141)
0620 18:48:17.923238 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.923240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.923248 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:48:17.923283 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.923284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.923294 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.923389 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:17.923555 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.924032 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.924945 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.925330 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.925423 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.925432 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.925578 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.925617 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:48:17.925843 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.923137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:40333 (local address 127.15.77.66:36389)
0620 18:48:17.923180 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.923182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.923335 (+ 153us) server_negotiation.cc:407] Connection header received
0620 18:48:17.923433 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.923434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.923453 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.923523 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:17.923686 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.924154 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.925077 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.925490 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.925578 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.925586 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.925744 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.925781 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:48:17.925858 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.923052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:36389 (local address 127.15.77.67:40333)
0620 18:48:17.923254 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:17.923271 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:17.923365 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:17.923529 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:17.923530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:17.923532 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:17.923616 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.923617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.924243 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.924244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.924985 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:17.924986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.925578 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.925579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:17.925652 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.925660 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 18:48:17.925735 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 18:48:17.925773 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":28,"thread_start_us":103,"threads_started":1}
I20260620 18:48:17.926250 432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d05e0980e040d8a4f77eca887da665" candidate_uuid: "492da6fe2729466797e4184e0a7b3a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c2092e39bf4245aa030043f3e35a53" is_pre_election: true
I20260620 18:48:17.926282 566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d05e0980e040d8a4f77eca887da665" candidate_uuid: "492da6fe2729466797e4184e0a7b3a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee66d316287047b08f721ffe2b91af8f" is_pre_election: true
I20260620 18:48:17.926429 432 raft_consensus.cc:2454] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492da6fe2729466797e4184e0a7b3a49 in term 0.
I20260620 18:48:17.926435 566 raft_consensus.cc:2454] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492da6fe2729466797e4184e0a7b3a49 in term 0.
I20260620 18:48:17.926663 636 leader_election.cc:302] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492da6fe2729466797e4184e0a7b3a49, ee66d316287047b08f721ffe2b91af8f; no voters:
I20260620 18:48:17.926795 769 raft_consensus.cc:2781] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:48:17.926849 769 raft_consensus.cc:482] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:48:17.926882 769 raft_consensus.cc:3037] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:17.927690 769 raft_consensus.cc:504] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 } } peers { permanent_uuid: "492da6fe2729466797e4184e0a7b3a49" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45761 } } peers { permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 } }
I20260620 18:48:17.927848 769 leader_election.cc:288] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [CANDIDATE]: Term 1 election: Requested vote from peers ee66d316287047b08f721ffe2b91af8f (127.15.77.66:36389), b4c2092e39bf4245aa030043f3e35a53 (127.15.77.65:33141)
I20260620 18:48:17.928018 566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d05e0980e040d8a4f77eca887da665" candidate_uuid: "492da6fe2729466797e4184e0a7b3a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee66d316287047b08f721ffe2b91af8f"
I20260620 18:48:17.928059 432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d05e0980e040d8a4f77eca887da665" candidate_uuid: "492da6fe2729466797e4184e0a7b3a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c2092e39bf4245aa030043f3e35a53"
I20260620 18:48:17.928117 432 raft_consensus.cc:3037] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:17.928138 566 raft_consensus.cc:3037] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:17.928932 432 raft_consensus.cc:2454] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492da6fe2729466797e4184e0a7b3a49 in term 1.
I20260620 18:48:17.929112 566 raft_consensus.cc:2454] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492da6fe2729466797e4184e0a7b3a49 in term 1.
I20260620 18:48:17.929126 635 leader_election.cc:302] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492da6fe2729466797e4184e0a7b3a49, b4c2092e39bf4245aa030043f3e35a53; no voters:
I20260620 18:48:17.929211 769 raft_consensus.cc:2781] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:17.929385 769 raft_consensus.cc:686] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 1 LEADER]: Becoming Leader. State: Replica: 492da6fe2729466797e4184e0a7b3a49, State: Running, Role: LEADER
I20260620 18:48:17.929510 769 consensus_queue.cc:237] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 } } peers { permanent_uuid: "492da6fe2729466797e4184e0a7b3a49" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45761 } } peers { permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 } }
I20260620 18:48:17.930385 32755 catalog_manager.cc:5495] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 492da6fe2729466797e4184e0a7b3a49 (127.15.77.67). New cstate: current_term: 1 leader_uuid: "492da6fe2729466797e4184e0a7b3a49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "492da6fe2729466797e4184e0a7b3a49" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 } health_report { overall_health: UNKNOWN } } }
W20260620 18:48:17.931989 613 tablet.cc:2369] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:48:17.948782 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.946262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.15.77.67:45761)
0620 18:48:17.946324 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:17.946327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:17.946337 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:48:17.946390 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:17.946390 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:17.946408 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:17.946497 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:17.946637 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.947129 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.947981 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:17.948173 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:17.948260 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:17.948434 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:17.948539 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:17.948636 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:17.948655 (+ 19us) server_negotiation.cc:299] Negotiation successful
0620 18:48:17.948676 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 18:48:17.950577 432 raft_consensus.cc:1261] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53 [term 1 FOLLOWER]: Refusing update from remote peer 492da6fe2729466797e4184e0a7b3a49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:17.950601 566 raft_consensus.cc:1261] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f [term 1 FOLLOWER]: Refusing update from remote peer 492da6fe2729466797e4184e0a7b3a49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:17.950948 769 consensus_queue.cc:1035] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4c2092e39bf4245aa030043f3e35a53" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 33141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:17.951115 773 consensus_queue.cc:1035] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee66d316287047b08f721ffe2b91af8f" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:17.954006 777 mvcc.cc:205] Tried to move back new op lower bound from 7298995396402896896 to 7298995396319977472. Current Snapshot: MvccSnapshot[applied={T|T < 7298995396402896896}]
I20260620 18:48:17.954337 778 mvcc.cc:205] Tried to move back new op lower bound from 7298995396402896896 to 7298995396319977472. Current Snapshot: MvccSnapshot[applied={T|T < 7298995396402896896}]
I20260620 18:48:17.955778 32754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:17.955860 32754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:17.956681 32754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c92fdbf5de884d1aa5c5896faad7f706: Sending DeleteTablet for 3 replicas of tablet 53d05e0980e040d8a4f77eca887da665
I20260620 18:48:17.957054 680 tablet_service.cc:1514] Processing DeleteTablet for tablet 53d05e0980e040d8a4f77eca887da665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:17 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:45506
I20260620 18:48:17.957072 412 tablet_service.cc:1514] Processing DeleteTablet for tablet 53d05e0980e040d8a4f77eca887da665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:17 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:59126
I20260620 18:48:17.957077 546 tablet_service.cc:1514] Processing DeleteTablet for tablet 53d05e0980e040d8a4f77eca887da665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:17 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:47066
I20260620 18:48:17.957413 786 tablet_replica.cc:330] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53: stopping tablet replica
I20260620 18:48:17.957530 786 raft_consensus.cc:2229] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:48:17.957638 786 raft_consensus.cc:2258] T 53d05e0980e040d8a4f77eca887da665 P b4c2092e39bf4245aa030043f3e35a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:17.957746 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 349
I20260620 18:48:17.959476 787 tablet_replica.cc:330] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: stopping tablet replica
I20260620 18:48:17.959600 787 raft_consensus.cc:2229] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:48:17.959704 787 raft_consensus.cc:2258] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:17.959861 788 tablet_replica.cc:330] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: stopping tablet replica
I20260620 18:48:17.960012 788 raft_consensus.cc:2229] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:48:17.960106 787 ts_tablet_manager.cc:1905] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:17.960166 788 raft_consensus.cc:2258] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:17.960618 788 ts_tablet_manager.cc:1905] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:17.962174 787 ts_tablet_manager.cc:1918] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:17.962301 787 log.cc:1196] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/53d05e0980e040d8a4f77eca887da665
I20260620 18:48:17.962576 788 ts_tablet_manager.cc:1918] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:17.962689 787 ts_tablet_manager.cc:1939] T 53d05e0980e040d8a4f77eca887da665 P 492da6fe2729466797e4184e0a7b3a49: Deleting consensus metadata
I20260620 18:48:17.962711 788 log.cc:1196] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/53d05e0980e040d8a4f77eca887da665
I20260620 18:48:17.962968 788 ts_tablet_manager.cc:1939] T 53d05e0980e040d8a4f77eca887da665 P ee66d316287047b08f721ffe2b91af8f: Deleting consensus metadata
I20260620 18:48:17.963162 32741 catalog_manager.cc:4841] TS 492da6fe2729466797e4184e0a7b3a49 (127.15.77.67:45761): tablet 53d05e0980e040d8a4f77eca887da665 (table test-table [id=a0a08cb7c08943b58a6eb04add8e5916]) successfully deleted
I20260620 18:48:17.963444 32742 catalog_manager.cc:4841] TS ee66d316287047b08f721ffe2b91af8f (127.15.77.66:36389): tablet 53d05e0980e040d8a4f77eca887da665 (table test-table [id=a0a08cb7c08943b58a6eb04add8e5916]) successfully deleted
W20260620 18:48:17.964893 32741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:33141 (error 108)
I20260620 18:48:17.965404 761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:17.965272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:33141 (local address 127.0.0.1:59132)
0620 18:48:17.965327 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:17.965364 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:33141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 18:48:17.965452 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 483
W20260620 18:48:17.970981 32741 catalog_manager.cc:4584] TS b4c2092e39bf4245aa030043f3e35a53 (127.15.77.65:33141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 53d05e0980e040d8a4f77eca887da665: Network error: Client connection negotiation failed: client connection to 127.15.77.65:33141: connect: Connection refused (error 111)
I20260620 18:48:17.971220 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 617
I20260620 18:48:17.975764 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 32723
2026-06-20T18:48:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1982 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-20T18:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:17Z Disabled control of system clock
I20260620 18:48:18.003262 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:33455
--webserver_interface=127.15.77.126
--webserver_port=0
--builtin_ntp_servers=127.15.77.84:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:33455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 18:48:18.086148 793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:18.086385 793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:18.086411 793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:18.087687 793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:18.087731 793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:18.087744 793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:18.087756 793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:18.087823 793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:18.089171 793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:33455
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:33455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:18.089443 793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:18.089756 793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:18.094553 801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:18.094568 799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:18.094576 798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:18.094803 793 server_base.cc:1034] running on GCE node
I20260620 18:48:18.095036 793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:18.095388 793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:18.096546 793 hybrid_clock.cc:648] HybridClock initialized: now 1781981298096543 us; error 38 us; skew 500 ppm
I20260620 18:48:18.097707 793 webserver.cc:460] Webserver started at http://127.15.77.126:37157/ using document root <none> and password file <none>
I20260620 18:48:18.097906 793 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:18.097947 793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:18.098038 793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:18.099040 793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "20e12ad1591d4b32a6d3566fd1641f55"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "732dbc462ab9d007b860ba4e20fceec2"
server_key_iv: "d206a820877607167b071c86e18ccce8"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.099414 793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "20e12ad1591d4b32a6d3566fd1641f55"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "732dbc462ab9d007b860ba4e20fceec2"
server_key_iv: "d206a820877607167b071c86e18ccce8"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.100654 793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:18.101302 807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:18.101495 793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:48:18.101604 793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "20e12ad1591d4b32a6d3566fd1641f55"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "732dbc462ab9d007b860ba4e20fceec2"
server_key_iv: "d206a820877607167b071c86e18ccce8"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.101660 793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:18.108731 793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:18.108970 793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:18.109068 793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:18.112775 793 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:33455
I20260620 18:48:18.112860 859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:33455 every 8 connection(s)
I20260620 18:48:18.113173 793 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:18.114236 860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:18.116577 860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55: Bootstrap starting.
I20260620 18:48:18.117177 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 793
I20260620 18:48:18.117234 860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:18.117276 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:18.117488 15669 external_mini_cluster.cc:1428] Setting key 5907966c0093fa2d924a90640ad6c4e8
I20260620 18:48:18.117556 860 log.cc:824] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:18.118661 860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55: No bootstrap required, opened a new log
I20260620 18:48:18.120935 860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e12ad1591d4b32a6d3566fd1641f55" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33455 } }
I20260620 18:48:18.121124 860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:18.121157 860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e12ad1591d4b32a6d3566fd1641f55, State: Initialized, Role: FOLLOWER
I20260620 18:48:18.121281 860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e12ad1591d4b32a6d3566fd1641f55" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33455 } }
I20260620 18:48:18.121374 860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:18.121399 860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:18.121471 860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:18.122121 860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e12ad1591d4b32a6d3566fd1641f55" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33455 } }
I20260620 18:48:18.122263 860 leader_election.cc:302] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20e12ad1591d4b32a6d3566fd1641f55; no voters:
I20260620 18:48:18.122462 860 leader_election.cc:288] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:18.122581 865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:18.122748 860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:18.122782 865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [term 1 LEADER]: Becoming Leader. State: Replica: 20e12ad1591d4b32a6d3566fd1641f55, State: Running, Role: LEADER
I20260620 18:48:18.123023 865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e12ad1591d4b32a6d3566fd1641f55" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33455 } }
I20260620 18:48:18.123468 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.117973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49688 (local address 127.15.77.126:33455)
0620 18:48:18.118192 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.118202 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.118236 (+ 34us) server_negotiation.cc:407] Connection header received
0620 18:48:18.118717 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.118727 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.118806 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.118950 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:18.119199 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.120129 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.121316 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.121821 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.121909 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.122674 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:18.122679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:18.122679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:18.122894 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:18.122938 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:18.122954 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:18.123017 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.123124 (+ 107us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.123204 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260620 18:48:18.123497 867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20e12ad1591d4b32a6d3566fd1641f55. Latest consensus state: current_term: 1 leader_uuid: "20e12ad1591d4b32a6d3566fd1641f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e12ad1591d4b32a6d3566fd1641f55" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33455 } } }
I20260620 18:48:18.123590 867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:18.124032 866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20e12ad1591d4b32a6d3566fd1641f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e12ad1591d4b32a6d3566fd1641f55" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33455 } } }
I20260620 18:48:18.124121 866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55 [sys.catalog]: This master's current role is: LEADER
W20260620 18:48:18.124794 881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:18.124838 881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:18.124897 874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:18.125087 874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:18.126508 874 catalog_manager.cc:1347] Generated new cluster ID: 27012d28f1ce45b79074231c7beeb8b1
I20260620 18:48:18.126586 874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:18.145141 874 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:18.145907 874 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:18.152333 874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55: Generated new TSK 0
I20260620 18:48:18.152531 874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:18.155845 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:33455
--builtin_ntp_servers=127.15.77.84:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 18:48:18.261710 884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:18.262017 884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:18.262053 884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:18.264201 884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:18.264258 884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:18.264302 884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:48:18.266700 884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:33455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:18.266984 884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:18.267385 884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:18.272979 889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:18.272986 890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:18.273178 884 server_base.cc:1034] running on GCE node
W20260620 18:48:18.273293 892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:18.273505 884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:18.273803 884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:18.274969 884 hybrid_clock.cc:648] HybridClock initialized: now 1781981298274953 us; error 33 us; skew 500 ppm
I20260620 18:48:18.276321 884 webserver.cc:460] Webserver started at http://127.15.77.65:35957/ using document root <none> and password file <none>
I20260620 18:48:18.276578 884 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:18.276646 884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:18.276788 884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:18.277916 884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "fcbd0265041b053788661536ebebebf8"
server_key_iv: "2cd021424a120f3fb0cffba4f0a0b280"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.278297 884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "fcbd0265041b053788661536ebebebf8"
server_key_iv: "2cd021424a120f3fb0cffba4f0a0b280"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.279721 884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:48:18.280555 898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:18.280736 884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:18.280859 884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "fcbd0265041b053788661536ebebebf8"
server_key_iv: "2cd021424a120f3fb0cffba4f0a0b280"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.280947 884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:18.288668 884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:18.288964 884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:18.289110 884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:18.289337 884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:18.289649 884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:18.289708 884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:18.289750 884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:18.289778 884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:18.295462 884 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:40133
I20260620 18:48:18.295497 1011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:40133 every 8 connection(s)
I20260620 18:48:18.295868 884 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:48:18.301137 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.296579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:51545 (local address 127.15.77.126:33455)
0620 18:48:18.296648 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.296651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.296923 (+ 272us) server_negotiation.cc:407] Connection header received
0620 18:48:18.297396 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.297397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.297408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.297497 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:18.298222 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.298737 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.299759 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.299946 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.300000 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.300856 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:18.300857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:18.300858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:18.300862 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:18.300877 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:18.300880 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:18.300934 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.301017 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.301040 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 18:48:18.301292 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 884
I20260620 18:48:18.301331 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.296573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:33455 (local address 127.15.77.65:51545)
0620 18:48:18.296780 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:18.296830 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:18.297315 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:18.297574 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:18.297578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:18.297586 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:18.298149 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.298154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.298812 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.298813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.299687 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.299689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.300026 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.300027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.300132 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.300703 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:48:18.300704 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:18.300732 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:48:18.300755 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:18.300945 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:18.300949 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:18.300951 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:48:18.301009 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 18:48:18.301106 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260620 18:48:18.301409 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:48:18.301607 15669 external_mini_cluster.cc:1428] Setting key d697284f2e312f1da24c3f1cc1c1c1d2
I20260620 18:48:18.302040 1012 heartbeater.cc:347] Connected to a master server at 127.15.77.126:33455
I20260620 18:48:18.302124 1012 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:18.302340 1012 heartbeater.cc:511] Master 127.15.77.126:33455 requested a full tablet report, sending...
I20260620 18:48:18.302636 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:33455
--builtin_ntp_servers=127.15.77.84:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 18:48:18.302915 824 ts_manager.cc:194] Registered new tserver with Master: 0e9b5a49cc0b4889ada88d12fa0b1d7d (127.15.77.65:40133)
I20260620 18:48:18.303874 824 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.65:51545
W20260620 18:48:18.384068 1015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:18.384270 1015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:18.384291 1015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:18.385663 1015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:18.385720 1015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:18.385876 1015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:48:18.387548 1015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:33455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:18.387811 1015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:18.388129 1015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:18.393225 1023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:18.393235 1020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:18.393235 1021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:18.393471 1015 server_base.cc:1034] running on GCE node
I20260620 18:48:18.393731 1015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:18.393961 1015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:18.395116 1015 hybrid_clock.cc:648] HybridClock initialized: now 1781981298395104 us; error 28 us; skew 500 ppm
I20260620 18:48:18.396389 1015 webserver.cc:460] Webserver started at http://127.15.77.66:42857/ using document root <none> and password file <none>
I20260620 18:48:18.396644 1015 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:18.396692 1015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:18.396822 1015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:18.397887 1015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "a28924667b3f4ff89a137ab48856b3a2"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "30bc1aff3ca8f29a1027d0c0b3d74f04"
server_key_iv: "bf3d9ad5615aee3303b6cb66c23c232a"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.398248 1015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "a28924667b3f4ff89a137ab48856b3a2"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "30bc1aff3ca8f29a1027d0c0b3d74f04"
server_key_iv: "bf3d9ad5615aee3303b6cb66c23c232a"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.399652 1015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:48:18.400442 1029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:18.400650 1015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:18.400758 1015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "a28924667b3f4ff89a137ab48856b3a2"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "30bc1aff3ca8f29a1027d0c0b3d74f04"
server_key_iv: "bf3d9ad5615aee3303b6cb66c23c232a"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.400835 1015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:18.410342 1015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:18.410634 1015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:18.410779 1015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:18.410997 1015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:18.411355 1015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:18.411422 1015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:18.411465 1015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:18.411491 1015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:18.417127 1015 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:38581
I20260620 18:48:18.417167 1142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:38581 every 8 connection(s)
I20260620 18:48:18.417501 1015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:48:18.417656 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 1015
I20260620 18:48:18.417771 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:48:18.418008 15669 external_mini_cluster.cc:1428] Setting key 1a9630d51682d8b03a0dfaea99fd652e
I20260620 18:48:18.418916 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:33455
--builtin_ntp_servers=127.15.77.84:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 18:48:18.423389 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.418248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:37567 (local address 127.15.77.126:33455)
0620 18:48:18.418300 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.418302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.418542 (+ 240us) server_negotiation.cc:407] Connection header received
0620 18:48:18.419105 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.419106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.419120 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.419157 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:18.419923 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.420551 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.421758 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.422088 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.422165 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.423039 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:18.423041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:18.423041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:18.423046 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:18.423065 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:18.423070 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:18.423133 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.423204 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.423233 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:48:18.423596 1145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.418236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:33455 (local address 127.15.77.66:37567)
0620 18:48:18.418403 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:18.418441 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:18.418963 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:18.419302 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:18.419306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:18.419339 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:18.419836 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.419840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.420653 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.420654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.421669 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.421672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.422176 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.422177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.422284 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.422889 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:48:18.422889 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:18.422918 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:48:18.422939 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:18.423139 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:18.423144 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:18.423147 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:48:18.423201 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 18:48:18.423278 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260620 18:48:18.424116 1143 heartbeater.cc:347] Connected to a master server at 127.15.77.126:33455
I20260620 18:48:18.424203 1143 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:18.424392 1143 heartbeater.cc:511] Master 127.15.77.126:33455 requested a full tablet report, sending...
I20260620 18:48:18.424784 824 ts_manager.cc:194] Registered new tserver with Master: a28924667b3f4ff89a137ab48856b3a2 (127.15.77.66:38581)
I20260620 18:48:18.425323 824 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.66:37567
W20260620 18:48:18.501262 1146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:18.501461 1146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:18.501482 1146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:18.502555 1146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:18.502588 1146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:18.502614 1146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:48:18.504164 1146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:33455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:18.504350 1146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:18.504703 1146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:18.509714 1152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:18.509707 1151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:18.509708 1154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:18.509757 1146 server_base.cc:1034] running on GCE node
I20260620 18:48:18.510159 1146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:18.510416 1146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:18.511574 1146 hybrid_clock.cc:648] HybridClock initialized: now 1781981298511571 us; error 38 us; skew 500 ppm
I20260620 18:48:18.512889 1146 webserver.cc:460] Webserver started at http://127.15.77.67:39805/ using document root <none> and password file <none>
I20260620 18:48:18.513157 1146 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:18.513234 1146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:18.513388 1146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:18.514562 1146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "13f80c8fdb1247928f9e4a4233e6185b"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "b1f0eba9b38544ddfad5013ef3d3f886"
server_key_iv: "4f25c0afcc29005f0856e2f5fe5045e8"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.514925 1146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "13f80c8fdb1247928f9e4a4233e6185b"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "b1f0eba9b38544ddfad5013ef3d3f886"
server_key_iv: "4f25c0afcc29005f0856e2f5fe5045e8"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.516270 1146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:48:18.516999 1160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:18.517163 1146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:48:18.517263 1146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "13f80c8fdb1247928f9e4a4233e6185b"
format_stamp: "Formatted at 2026-06-20 18:48:18 on dist-test-slave-tvl9"
server_key: "b1f0eba9b38544ddfad5013ef3d3f886"
server_key_iv: "4f25c0afcc29005f0856e2f5fe5045e8"
server_key_version: "encryptionkey@0"
I20260620 18:48:18.517345 1146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:18.529395 1146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:18.529716 1146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:18.529875 1146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:18.530112 1146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:18.530441 1146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:18.530495 1146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:18.530544 1146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:18.530575 1146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:18.536273 1146 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:33991
I20260620 18:48:18.536319 1273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:33991 every 8 connection(s)
I20260620 18:48:18.536644 1146 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:48:18.541501 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.537371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:54709 (local address 127.15.77.126:33455)
0620 18:48:18.537453 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.537455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.537657 (+ 202us) server_negotiation.cc:407] Connection header received
0620 18:48:18.538113 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.538114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.538124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.538177 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:18.538800 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.539282 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.540292 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.540475 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.540528 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.541274 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:18.541275 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:18.541276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:18.541280 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:18.541293 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:18.541296 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:18.541339 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.541421 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.541441 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 18:48:18.541734 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.537334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:33455 (local address 127.15.77.67:54709)
0620 18:48:18.537532 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:18.537566 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:18.538030 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:18.538227 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:18.538230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:18.538237 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:18.538732 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.538736 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.539429 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.539429 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.540216 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.540218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.540546 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.540547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.540654 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.541145 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:48:18.541146 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:18.541174 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:48:18.541195 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:18.541349 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:18.541353 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:18.541355 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:48:18.541405 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 18:48:18.541488 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260620 18:48:18.542171 1274 heartbeater.cc:347] Connected to a master server at 127.15.77.126:33455
I20260620 18:48:18.542275 1274 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:18.542491 1274 heartbeater.cc:511] Master 127.15.77.126:33455 requested a full tablet report, sending...
I20260620 18:48:18.542872 824 ts_manager.cc:194] Registered new tserver with Master: 13f80c8fdb1247928f9e4a4233e6185b (127.15.77.67:33991)
I20260620 18:48:18.543246 824 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.67:54709
I20260620 18:48:18.544154 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 1146
I20260620 18:48:18.544222 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:48:18.544414 15669 external_mini_cluster.cc:1428] Setting key 9bdac18399af6ef7d0ff2b14d9f9d2ac
I20260620 18:48:18.545168 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:48:18.549108 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.546540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49698 (local address 127.15.77.126:33455)
0620 18:48:18.546579 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.546581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.546686 (+ 105us) server_negotiation.cc:407] Connection header received
0620 18:48:18.546723 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.546724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.546732 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.546791 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:18.546905 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.547406 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.548116 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.548288 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.548336 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.548934 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:18.548935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:18.548936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:18.548939 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:18.548952 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:18.548956 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:18.548997 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.549034 (+ 37us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.549052 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 18:48:18.550751 824 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:48:18.552407 824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:48:18.558751 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.554494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:33991 (local address 127.0.0.1:45162)
0620 18:48:18.554802 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:18.554844 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:18.554921 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:18.555146 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:18.555149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:18.555151 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:18.555237 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.555239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.556613 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.556614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.557659 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.557666 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.558472 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.558472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.558524 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.558534 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 18:48:18.558611 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 18:48:18.558667 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":20,"thread_start_us":67,"threads_started":1}
I20260620 18:48:18.558751 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.554628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.15.77.67:33991)
0620 18:48:18.554778 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.554782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.554904 (+ 122us) server_negotiation.cc:407] Connection header received
0620 18:48:18.555017 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.555019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.555036 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.555318 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:18.555394 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.556521 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.557735 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.558376 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.558470 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.558556 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.558626 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.558665 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260620 18:48:18.558751 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.554579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.15.77.66:38581)
0620 18:48:18.554731 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.554737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.554910 (+ 173us) server_negotiation.cc:407] Connection header received
0620 18:48:18.555053 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.555056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.555073 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.555130 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:18.555498 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.556222 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.557585 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.558342 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.558424 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.558524 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.558633 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.558669 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260620 18:48:18.559008 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.554505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:38581 (local address 127.0.0.1:42632)
0620 18:48:18.554811 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:18.554844 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:18.554940 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:18.555355 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:18.555356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:18.555358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:18.555406 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.555407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.556303 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.556304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.557515 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.557518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.558437 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.558438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.558516 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.558531 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 18:48:18.558602 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 18:48:18.558667 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260620 18:48:18.559624 1208 tablet_service.cc:1467] Processing CreateTablet for tablet 1fbda71e184c465893906dbc72a7c93d (DEFAULT_TABLE table=test-table [id=df05d208e5764ab7a10d41e2da7d1de1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:18.559628 1077 tablet_service.cc:1467] Processing CreateTablet for tablet 1fbda71e184c465893906dbc72a7c93d (DEFAULT_TABLE table=test-table [id=df05d208e5764ab7a10d41e2da7d1de1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:18.559957 1282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.554711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:40133 (local address 127.0.0.1:60578)
0620 18:48:18.555045 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:18.555055 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:18.555092 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:18.555255 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:18.555256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:18.555258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:18.555299 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.555301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.557731 (+ 2430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.557731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.558356 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.558357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.559743 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.559744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.559800 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.559804 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:48:18.559855 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 18:48:18.559894 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":58,"threads_started":1}
I20260620 18:48:18.559993 1077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fbda71e184c465893906dbc72a7c93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:18.559993 1208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fbda71e184c465893906dbc72a7c93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:18.560112 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.554900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.15.77.65:40133)
0620 18:48:18.555093 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.555100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.555113 (+ 13us) server_negotiation.cc:407] Connection header received
0620 18:48:18.555151 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.555152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.555165 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.555240 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:18.555402 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.556511 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.558705 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.559644 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.559738 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.559869 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.559894 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.559929 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260620 18:48:18.560681 946 tablet_service.cc:1467] Processing CreateTablet for tablet 1fbda71e184c465893906dbc72a7c93d (DEFAULT_TABLE table=test-table [id=df05d208e5764ab7a10d41e2da7d1de1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:18.560976 946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fbda71e184c465893906dbc72a7c93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:18.563539 1288 tablet_bootstrap.cc:492] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d: Bootstrap starting.
I20260620 18:48:18.563956 1289 tablet_bootstrap.cc:492] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: Bootstrap starting.
I20260620 18:48:18.564218 1288 tablet_bootstrap.cc:654] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:18.564455 1289 tablet_bootstrap.cc:654] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:18.564555 1288 log.cc:824] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:18.564672 1289 log.cc:824] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:18.564992 1290 tablet_bootstrap.cc:492] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2: Bootstrap starting.
I20260620 18:48:18.565303 1288 tablet_bootstrap.cc:492] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d: No bootstrap required, opened a new log
I20260620 18:48:18.565383 1288 ts_tablet_manager.cc:1397] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:48:18.565510 1290 tablet_bootstrap.cc:654] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:18.565722 1290 log.cc:824] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:18.566107 1289 tablet_bootstrap.cc:492] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: No bootstrap required, opened a new log
I20260620 18:48:18.566200 1289 ts_tablet_manager.cc:1397] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:48:18.566484 1290 tablet_bootstrap.cc:492] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2: No bootstrap required, opened a new log
I20260620 18:48:18.566545 1290 ts_tablet_manager.cc:1397] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:48:18.567677 1288 raft_consensus.cc:348] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 } } peers { permanent_uuid: "13f80c8fdb1247928f9e4a4233e6185b" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33991 } } peers { permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 } }
I20260620 18:48:18.567829 1288 raft_consensus.cc:374] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:18.567873 1288 raft_consensus.cc:729] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e9b5a49cc0b4889ada88d12fa0b1d7d, State: Initialized, Role: FOLLOWER
I20260620 18:48:18.568001 1288 consensus_queue.cc:260] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 } } peers { permanent_uuid: "13f80c8fdb1247928f9e4a4233e6185b" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33991 } } peers { permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 } }
I20260620 18:48:18.568286 1288 ts_tablet_manager.cc:1428] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:48:18.568420 1012 heartbeater.cc:503] Master 127.15.77.126:33455 was elected leader, sending a full tablet report...
I20260620 18:48:18.569026 1289 raft_consensus.cc:348] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 } } peers { permanent_uuid: "13f80c8fdb1247928f9e4a4233e6185b" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33991 } } peers { permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 } }
I20260620 18:48:18.569192 1289 raft_consensus.cc:374] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:18.569235 1289 raft_consensus.cc:729] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f80c8fdb1247928f9e4a4233e6185b, State: Initialized, Role: FOLLOWER
I20260620 18:48:18.569269 1290 raft_consensus.cc:348] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 } } peers { permanent_uuid: "13f80c8fdb1247928f9e4a4233e6185b" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33991 } } peers { permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 } }
I20260620 18:48:18.569361 1289 consensus_queue.cc:260] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 } } peers { permanent_uuid: "13f80c8fdb1247928f9e4a4233e6185b" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33991 } } peers { permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 } }
I20260620 18:48:18.569410 1290 raft_consensus.cc:374] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:18.569444 1290 raft_consensus.cc:729] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a28924667b3f4ff89a137ab48856b3a2, State: Initialized, Role: FOLLOWER
I20260620 18:48:18.569603 1289 ts_tablet_manager.cc:1428] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260620 18:48:18.569576 1290 consensus_queue.cc:260] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 } } peers { permanent_uuid: "13f80c8fdb1247928f9e4a4233e6185b" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33991 } } peers { permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 } }
I20260620 18:48:18.569718 1274 heartbeater.cc:503] Master 127.15.77.126:33455 was elected leader, sending a full tablet report...
I20260620 18:48:18.569830 1290 ts_tablet_manager.cc:1428] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:48:18.569877 1143 heartbeater.cc:503] Master 127.15.77.126:33455 was elected leader, sending a full tablet report...
W20260620 18:48:18.668197 1144 tablet.cc:2369] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:48:18.761564 1295 raft_consensus.cc:482] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:18.761842 1295 raft_consensus.cc:504] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 } } peers { permanent_uuid: "13f80c8fdb1247928f9e4a4233e6185b" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33991 } } peers { permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 } }
I20260620 18:48:18.762270 1295 leader_election.cc:288] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a28924667b3f4ff89a137ab48856b3a2 (127.15.77.66:38581), 0e9b5a49cc0b4889ada88d12fa0b1d7d (127.15.77.65:40133)
I20260620 18:48:18.765455 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.762653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:40133 (local address 127.15.77.67:37223)
0620 18:48:18.762823 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:18.762834 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:18.762911 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:18.763048 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:18.763048 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:18.763050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:18.763116 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.763117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.763782 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.763784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.764616 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.764619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.765167 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.765169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.765248 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.765252 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:48:18.765310 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 18:48:18.765347 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":5,"thread_start_us":108,"threads_started":1}
I20260620 18:48:18.765453 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.762767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:37223 (local address 127.15.77.65:40133)
0620 18:48:18.762816 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.762822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.762903 (+ 81us) server_negotiation.cc:407] Connection header received
0620 18:48:18.762975 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.762976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.762988 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.763044 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:18.763181 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.763693 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.764716 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.765077 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.765134 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.765141 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.765324 (+ 183us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.765371 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 18:48:18.765576 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.762619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:35185 (local address 127.15.77.66:38581)
0620 18:48:18.762691 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.762694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.762706 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:48:18.762741 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.762743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.762752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.762834 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:18.763049 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.763698 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.764735 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.765174 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.765246 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.765254 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.765447 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.765499 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 18:48:18.765596 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.762522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:38581 (local address 127.15.77.67:35185)
0620 18:48:18.762582 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:18.762595 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:18.762642 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:18.762859 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:18.762860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:18.762862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:18.762959 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.762962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.763810 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.763811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.764621 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:18.764623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.765269 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.765270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:18.765360 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.765363 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:48:18.765422 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 18:48:18.765461 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":41}
I20260620 18:48:18.765866 966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fbda71e184c465893906dbc72a7c93d" candidate_uuid: "13f80c8fdb1247928f9e4a4233e6185b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" is_pre_election: true
I20260620 18:48:18.765884 1097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fbda71e184c465893906dbc72a7c93d" candidate_uuid: "13f80c8fdb1247928f9e4a4233e6185b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a28924667b3f4ff89a137ab48856b3a2" is_pre_election: true
I20260620 18:48:18.766043 966 raft_consensus.cc:2454] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13f80c8fdb1247928f9e4a4233e6185b in term 0.
I20260620 18:48:18.766044 1097 raft_consensus.cc:2454] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13f80c8fdb1247928f9e4a4233e6185b in term 0.
I20260620 18:48:18.766274 1162 leader_election.cc:302] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13f80c8fdb1247928f9e4a4233e6185b, a28924667b3f4ff89a137ab48856b3a2; no voters:
I20260620 18:48:18.766394 1295 raft_consensus.cc:2781] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:48:18.766450 1295 raft_consensus.cc:482] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:48:18.766479 1295 raft_consensus.cc:3037] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:18.767166 1295 raft_consensus.cc:504] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 } } peers { permanent_uuid: "13f80c8fdb1247928f9e4a4233e6185b" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33991 } } peers { permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 } }
I20260620 18:48:18.767302 1295 leader_election.cc:288] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [CANDIDATE]: Term 1 election: Requested vote from peers a28924667b3f4ff89a137ab48856b3a2 (127.15.77.66:38581), 0e9b5a49cc0b4889ada88d12fa0b1d7d (127.15.77.65:40133)
I20260620 18:48:18.767540 966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fbda71e184c465893906dbc72a7c93d" candidate_uuid: "13f80c8fdb1247928f9e4a4233e6185b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d"
I20260620 18:48:18.767571 1097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fbda71e184c465893906dbc72a7c93d" candidate_uuid: "13f80c8fdb1247928f9e4a4233e6185b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a28924667b3f4ff89a137ab48856b3a2"
I20260620 18:48:18.767617 966 raft_consensus.cc:3037] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:18.767664 1097 raft_consensus.cc:3037] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:18.768361 966 raft_consensus.cc:2454] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13f80c8fdb1247928f9e4a4233e6185b in term 1.
I20260620 18:48:18.768370 1097 raft_consensus.cc:2454] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13f80c8fdb1247928f9e4a4233e6185b in term 1.
I20260620 18:48:18.768577 1162 leader_election.cc:302] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13f80c8fdb1247928f9e4a4233e6185b, a28924667b3f4ff89a137ab48856b3a2; no voters:
I20260620 18:48:18.768699 1295 raft_consensus.cc:2781] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:18.768883 1295 raft_consensus.cc:686] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 1 LEADER]: Becoming Leader. State: Replica: 13f80c8fdb1247928f9e4a4233e6185b, State: Running, Role: LEADER
I20260620 18:48:18.768997 1295 consensus_queue.cc:237] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 } } peers { permanent_uuid: "13f80c8fdb1247928f9e4a4233e6185b" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33991 } } peers { permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 } }
I20260620 18:48:18.769944 823 catalog_manager.cc:5495] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b reported cstate change: term changed from 0 to 1, leader changed from <none> to 13f80c8fdb1247928f9e4a4233e6185b (127.15.77.67). New cstate: current_term: 1 leader_uuid: "13f80c8fdb1247928f9e4a4233e6185b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13f80c8fdb1247928f9e4a4233e6185b" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 } health_report { overall_health: UNKNOWN } } }
W20260620 18:48:18.787166 1275 tablet.cc:2369] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:48:18.788380 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.786057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.15.77.67:33991)
0620 18:48:18.786112 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:18.786115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:18.786123 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:48:18.786163 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:18.786164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:18.786182 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:18.786242 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:18.786342 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.786804 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.787716 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:18.787909 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:18.787965 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:18.788101 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:18.788197 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:18.788258 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:18.788288 (+ 30us) server_negotiation.cc:299] Negotiation successful
0620 18:48:18.788307 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 18:48:18.790222 1097 raft_consensus.cc:1261] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2 [term 1 FOLLOWER]: Refusing update from remote peer 13f80c8fdb1247928f9e4a4233e6185b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:18.790222 966 raft_consensus.cc:1261] T 1fbda71e184c465893906dbc72a7c93d P 0e9b5a49cc0b4889ada88d12fa0b1d7d [term 1 FOLLOWER]: Refusing update from remote peer 13f80c8fdb1247928f9e4a4233e6185b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:18.790622 1298 consensus_queue.cc:1035] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a28924667b3f4ff89a137ab48856b3a2" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 38581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:18.790768 1295 consensus_queue.cc:1035] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e9b5a49cc0b4889ada88d12fa0b1d7d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:18.793629 1302 mvcc.cc:205] Tried to move back new op lower bound from 7298995399841779712 to 7298995399758602240. Current Snapshot: MvccSnapshot[applied={T|T < 7298995399841779712}]
I20260620 18:48:18.795573 824 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:18.795675 824 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:18.796464 824 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20e12ad1591d4b32a6d3566fd1641f55: Sending DeleteTablet for 3 replicas of tablet 1fbda71e184c465893906dbc72a7c93d
I20260620 18:48:18.796896 1077 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fbda71e184c465893906dbc72a7c93d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:18 UTC) from {username='slave'} at 127.0.0.1:42632
I20260620 18:48:18.796870 946 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fbda71e184c465893906dbc72a7c93d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:18 UTC) from {username='slave'} at 127.0.0.1:60578
I20260620 18:48:18.796900 1208 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fbda71e184c465893906dbc72a7c93d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:18 UTC) from {username='slave'} at 127.0.0.1:45162
I20260620 18:48:18.797160 1312 tablet_replica.cc:330] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: stopping tablet replica
I20260620 18:48:18.797264 1312 raft_consensus.cc:2229] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:48:18.797353 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 884
I20260620 18:48:18.797390 1312 raft_consensus.cc:2258] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:18.798249 1312 ts_tablet_manager.cc:1905] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:18.800371 1312 ts_tablet_manager.cc:1918] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:18.800452 1312 log.cc:1196] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/1fbda71e184c465893906dbc72a7c93d
I20260620 18:48:18.800710 1312 ts_tablet_manager.cc:1939] T 1fbda71e184c465893906dbc72a7c93d P 13f80c8fdb1247928f9e4a4233e6185b: Deleting consensus metadata
I20260620 18:48:18.801120 810 catalog_manager.cc:4841] TS 13f80c8fdb1247928f9e4a4233e6185b (127.15.77.67:33991): tablet 1fbda71e184c465893906dbc72a7c93d (table test-table [id=df05d208e5764ab7a10d41e2da7d1de1]) successfully deleted
I20260620 18:48:18.803445 1310 tablet_replica.cc:330] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2: stopping tablet replica
I20260620 18:48:18.803617 1310 raft_consensus.cc:2229] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:48:18.803750 1310 raft_consensus.cc:2258] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:18.804358 1310 ts_tablet_manager.cc:1905] T 1fbda71e184c465893906dbc72a7c93d P a28924667b3f4ff89a137ab48856b3a2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 18:48:18.805238 809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:40133 (error 108)
I20260620 18:48:18.805528 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 1015
I20260620 18:48:18.811158 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 1146
I20260620 18:48:18.811271 1282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.811098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:40133 (local address 127.0.0.1:60590)
0620 18:48:18.811161 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:18.811224 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:40133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 18:48:18.811467 809 catalog_manager.cc:4584] TS 0e9b5a49cc0b4889ada88d12fa0b1d7d (127.15.77.65:40133): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1fbda71e184c465893906dbc72a7c93d: Network error: Client connection negotiation failed: client connection to 127.15.77.65:40133: connect: Connection refused (error 111)
I20260620 18:48:18.811856 1282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:18.811377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:38581 (local address 127.0.0.1:42638)
0620 18:48:18.811799 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:18.811826 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.66:38581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404}
I20260620 18:48:18.815896 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 793
2026-06-20T18:48:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (839 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:18 dist-test-slave-tvl9 krb5kdc[1316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:18 dist-test-slave-tvl9 krb5kdc[1316](info): set up 2 sockets
Jun 20 18:48:18 dist-test-slave-tvl9 krb5kdc[1316](info): commencing operation
krb5kdc: starting...
W20260620 18:48:19.847107 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:19 dist-test-slave-tvl9 krb5kdc[1316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:19.967237 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34799
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:34799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:20.048444 1329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:20.048846 1329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:20.048892 1329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:20.050154 1329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:20.050233 1329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:20.050257 1329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:20.050276 1329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:20.050304 1329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:20.051923 1329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:34799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:20.052217 1329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:20.052459 1329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:20.057444 1334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:20.057595 1329 server_base.cc:1034] running on GCE node
W20260620 18:48:20.057453 1337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:20.057467 1335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:20.057938 1329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:20.058256 1329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:20.059396 1329 hybrid_clock.cc:648] HybridClock initialized: now 1781981300059373 us; error 39 us; skew 500 ppm
Jun 20 18:48:20 dist-test-slave-tvl9 krb5kdc[1316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981300, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:20.061915 1329 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:20.062362 1329 webserver.cc:460] Webserver started at http://127.15.77.126:35187/ using document root <none> and password file <none>
I20260620 18:48:20.062621 1329 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:20.062667 1329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:20.062827 1329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:20.063995 1329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "8812a83973b34d9d8f81184523f62e44"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "821a06e471426b7d9e4e02a05614759b"
server_key_iv: "670f90d7286961df7dc5960e68ca17bb"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.064354 1329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "8812a83973b34d9d8f81184523f62e44"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "821a06e471426b7d9e4e02a05614759b"
server_key_iv: "670f90d7286961df7dc5960e68ca17bb"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.065728 1329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:48:20.066514 1344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.066725 1329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:20.066818 1329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "8812a83973b34d9d8f81184523f62e44"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "821a06e471426b7d9e4e02a05614759b"
server_key_iv: "670f90d7286961df7dc5960e68ca17bb"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.066905 1329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:20.078725 1329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:20.081168 1329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:20.081329 1329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:20.085335 1329 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:34799
I20260620 18:48:20.085389 1396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:34799 every 8 connection(s)
I20260620 18:48:20.085740 1329 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:20.086745 1397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:20.089083 1397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44: Bootstrap starting.
I20260620 18:48:20.089637 1397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:20.089880 1397 log.cc:824] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:20.090569 1397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44: No bootstrap required, opened a new log
I20260620 18:48:20.091985 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 1329
I20260620 18:48:20.092110 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:20.092288 15669 external_mini_cluster.cc:1428] Setting key a8302cce5b684157b464288a7c3e5fb1
I20260620 18:48:20.092316 1397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8812a83973b34d9d8f81184523f62e44" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34799 } }
I20260620 18:48:20.092536 1397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:20.092581 1397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8812a83973b34d9d8f81184523f62e44, State: Initialized, Role: FOLLOWER
I20260620 18:48:20.092746 1397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8812a83973b34d9d8f81184523f62e44" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34799 } }
I20260620 18:48:20.092880 1397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:20.092947 1397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:20.093045 1397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:20.094127 1397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8812a83973b34d9d8f81184523f62e44" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34799 } }
I20260620 18:48:20.094233 1397 leader_election.cc:302] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8812a83973b34d9d8f81184523f62e44; no voters:
I20260620 18:48:20.094389 1397 leader_election.cc:288] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:20.094456 1402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:20.094645 1397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:20.094650 1402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [term 1 LEADER]: Becoming Leader. State: Replica: 8812a83973b34d9d8f81184523f62e44, State: Running, Role: LEADER
I20260620 18:48:20.094854 1402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8812a83973b34d9d8f81184523f62e44" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34799 } }
I20260620 18:48:20.095250 1404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8812a83973b34d9d8f81184523f62e44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8812a83973b34d9d8f81184523f62e44" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34799 } } }
I20260620 18:48:20.095355 1404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:20.095539 1404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8812a83973b34d9d8f81184523f62e44. Latest consensus state: current_term: 1 leader_uuid: "8812a83973b34d9d8f81184523f62e44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8812a83973b34d9d8f81184523f62e44" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34799 } } }
I20260620 18:48:20.095597 1404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44 [sys.catalog]: This master's current role is: LEADER
W20260620 18:48:20.096572 1417 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:20.096652 1417 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:20.096715 1411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:20.096895 1411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:20.098526 1411 catalog_manager.cc:1347] Generated new cluster ID: 9373c5e24d7c43869354ae925c7f9a78
I20260620 18:48:20.098577 1411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:48:20 dist-test-slave-tvl9 krb5kdc[1316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:20.103035 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.092841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36876 (local address 127.15.77.126:34799)
0620 18:48:20.093046 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.093056 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.093090 (+ 34us) server_negotiation.cc:407] Connection header received
0620 18:48:20.093216 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.093223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.093279 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.093437 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:20.094039 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.095455 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.096553 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.097170 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.097291 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.099831 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:20.099835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:20.099835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:20.099847 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:20.101534 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:20.101905 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:20.101908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:20.101908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:20.102008 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:20.102213 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:20.102214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:20.102215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:20.102459 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:20.102633 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.102705 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.102846 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":113,"threads_started":1}
I20260620 18:48:20.116613 1411 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:20.117149 1411 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:20.122756 1411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44: Generated new TSK 0
I20260620 18:48:20.122967 1411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:48:20.152114 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34799
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:20.234978 1423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:20.235196 1423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:20.235219 1423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:20.236505 1423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:20.236545 1423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:20.236573 1423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:48:20.238051 1423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:20.238301 1423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:20.238543 1423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:20.243664 1429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:20.243657 1428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:20.243865 1423 server_base.cc:1034] running on GCE node
W20260620 18:48:20.243657 1431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:20.244221 1423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:20.244465 1423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:20.245610 1423 hybrid_clock.cc:648] HybridClock initialized: now 1781981300245578 us; error 46 us; skew 500 ppm
Jun 20 18:48:20 dist-test-slave-tvl9 krb5kdc[1316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981300, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:20.248293 1423 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:48:20.248899 1423 webserver.cc:460] Webserver started at http://127.15.77.65:43915/ using document root <none> and password file <none>
I20260620 18:48:20.249168 1423 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:20.249223 1423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:20.249346 1423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:20.250810 1423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "0a2eb2033c4749a8ba034ae4fece480b"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "0df97a3b7f1bf9842c6dd7af1a2ba45f"
server_key_iv: "4d07aeec504041821553144c9c760f56"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.251271 1423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "0a2eb2033c4749a8ba034ae4fece480b"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "0df97a3b7f1bf9842c6dd7af1a2ba45f"
server_key_iv: "4d07aeec504041821553144c9c760f56"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.253225 1423 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260620 18:48:20.255518 1438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.255707 1423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 18:48:20.255827 1423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "0a2eb2033c4749a8ba034ae4fece480b"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "0df97a3b7f1bf9842c6dd7af1a2ba45f"
server_key_iv: "4d07aeec504041821553144c9c760f56"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.255934 1423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:20.280371 1423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:20.282964 1423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:20.283118 1423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:20.283406 1423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:20.283834 1423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:20.283885 1423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.283947 1423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:20.283974 1423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.291193 1423 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:42983
I20260620 18:48:20.291276 1551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:42983 every 8 connection(s)
I20260620 18:48:20.291632 1423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:48:20.297251 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 1423
I20260620 18:48:20.297379 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:48:20.297624 15669 external_mini_cluster.cc:1428] Setting key 27d350115531d3ae0647fd8530018e75
Jun 20 18:48:20 dist-test-slave-tvl9 krb5kdc[1316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981300, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:20.302380 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.292440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:58563 (local address 127.15.77.126:34799)
0620 18:48:20.292517 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.292520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.292707 (+ 187us) server_negotiation.cc:407] Connection header received
0620 18:48:20.293244 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.293245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.293257 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.293321 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:20.294695 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.295546 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.296808 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.297020 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.297142 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.299307 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:20.299309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:20.299309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:20.299352 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:20.300985 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:20.301377 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:20.301378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:20.301379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:20.301444 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:20.301675 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:20.301676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:20.301676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:20.301836 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:20.301916 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.302172 (+ 256us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.302270 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 18:48:20.302407 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.292392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:34799 (local address 127.15.77.65:58563)
0620 18:48:20.292571 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:20.292623 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:20.293135 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:20.293456 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:20.293461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:20.294240 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:20.294543 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.294550 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.295647 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.295671 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.296722 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.296725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.297152 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.297153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.297256 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.297839 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:20.297848 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:20.299217 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:20.301076 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:20.301083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:20.301085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:20.301301 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:20.301510 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:20.301511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:20.301511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:20.301610 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:20.301904 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:20.301908 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:20.302030 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 18:48:20.302110 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 18:48:20.302237 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260620 18:48:20.302963 1552 heartbeater.cc:347] Connected to a master server at 127.15.77.126:34799
I20260620 18:48:20.303105 1552 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:20.303305 1552 heartbeater.cc:511] Master 127.15.77.126:34799 requested a full tablet report, sending...
I20260620 18:48:20.303941 1361 ts_manager.cc:194] Registered new tserver with Master: 0a2eb2033c4749a8ba034ae4fece480b (127.15.77.65:42983)
I20260620 18:48:20.304939 1361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:58563
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:48:20.324168 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34799
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:20.405792 1557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:20.406064 1557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:20.406111 1557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:20.407289 1557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:20.407395 1557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:20.407449 1557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:48:20.409025 1557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:20.409307 1557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:20.409548 1557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:20.414737 1562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:20.414777 1563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:20.414763 1565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:20.414995 1557 server_base.cc:1034] running on GCE node
I20260620 18:48:20.415221 1557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:20.415500 1557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:20.416676 1557 hybrid_clock.cc:648] HybridClock initialized: now 1781981300416664 us; error 51 us; skew 500 ppm
Jun 20 18:48:20 dist-test-slave-tvl9 krb5kdc[1316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981300, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:20.419173 1557 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:48:20.419713 1557 webserver.cc:460] Webserver started at http://127.15.77.66:33351/ using document root <none> and password file <none>
I20260620 18:48:20.419945 1557 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:20.419999 1557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:20.420147 1557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:20.421208 1557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "ca653b78d675426e81045bbf2d276c4d"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "c01fa03a80eecf1d85587367e1315c97"
server_key_iv: "1600a9084de0ab42b938d48e007b7a25"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.421574 1557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "ca653b78d675426e81045bbf2d276c4d"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "c01fa03a80eecf1d85587367e1315c97"
server_key_iv: "1600a9084de0ab42b938d48e007b7a25"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.422901 1557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:20.423738 1572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.423938 1557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 18:48:20.424041 1557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "ca653b78d675426e81045bbf2d276c4d"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "c01fa03a80eecf1d85587367e1315c97"
server_key_iv: "1600a9084de0ab42b938d48e007b7a25"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.424115 1557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:20.442207 1557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:20.444327 1557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:20.444438 1557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:20.444715 1557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:20.445053 1557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:20.445087 1557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.445132 1557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:20.445160 1557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.451013 1557 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:35319
I20260620 18:48:20.451054 1685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:35319 every 8 connection(s)
I20260620 18:48:20.451428 1557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
Jun 20 18:48:20 dist-test-slave-tvl9 krb5kdc[1316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981300, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:20.459463 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 1557
I20260620 18:48:20.459592 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:48:20.459825 15669 external_mini_cluster.cc:1428] Setting key ea358a10aac4e537af72594dcb1b76bd
I20260620 18:48:20.460402 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.452152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:42193 (local address 127.15.77.126:34799)
0620 18:48:20.452206 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.452209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.452435 (+ 226us) server_negotiation.cc:407] Connection header received
0620 18:48:20.452913 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.452915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.452926 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.452994 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:20.453867 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.454431 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.455429 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.455618 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.455671 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.457635 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:20.457636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:20.457636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:20.457645 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:20.458883 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:20.459296 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:20.459297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:20.459297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:20.459396 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:20.459699 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:20.459701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:20.459701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:20.459848 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:20.459921 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.460236 (+ 315us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.460318 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:48:20.460613 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.452134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:34799 (local address 127.15.77.66:42193)
0620 18:48:20.452290 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:20.452327 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:20.452830 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:20.453049 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:20.453052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:20.453605 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:20.453792 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.453796 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.454520 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.454521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.455302 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.455304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.455689 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.455690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.455792 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.456219 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:20.456227 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:20.457533 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:20.458981 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:20.458987 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:20.458989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:20.459217 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:20.459475 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:20.459476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:20.459477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:20.459620 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:20.459921 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:20.459928 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:20.460068 (+ 140us) client_negotiation.cc:769] Sending connection context
0620 18:48:20.460203 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 18:48:20.460346 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
I20260620 18:48:20.461099 1686 heartbeater.cc:347] Connected to a master server at 127.15.77.126:34799
I20260620 18:48:20.461201 1686 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:20.461395 1686 heartbeater.cc:511] Master 127.15.77.126:34799 requested a full tablet report, sending...
I20260620 18:48:20.461791 1361 ts_manager.cc:194] Registered new tserver with Master: ca653b78d675426e81045bbf2d276c4d (127.15.77.66:35319)
I20260620 18:48:20.462306 1361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:42193
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:48:20.484537 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34799
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:20.565784 1691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:20.566059 1691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:20.566103 1691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:20.567428 1691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:20.567495 1691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:20.567541 1691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:48:20.569015 1691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:34799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:20.569291 1691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:20.569542 1691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:20.574745 1696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:20.574725 1697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:20.574721 1699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:20.574800 1691 server_base.cc:1034] running on GCE node
I20260620 18:48:20.575147 1691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:20.575410 1691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:20.576602 1691 hybrid_clock.cc:648] HybridClock initialized: now 1781981300576593 us; error 41 us; skew 500 ppm
Jun 20 18:48:20 dist-test-slave-tvl9 krb5kdc[1316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981300, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:20.579144 1691 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:48:20.579720 1691 webserver.cc:460] Webserver started at http://127.15.77.67:45285/ using document root <none> and password file <none>
I20260620 18:48:20.579995 1691 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:20.580045 1691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:20.580178 1691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:20.581224 1691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "d5553e3d018c4cffadfee3b07d1dd138"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "ab83ae7fb18c9dbea61fba7ba5cfb1c1"
server_key_iv: "2de645266c46c3dea9c95efaaf551cc0"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.581575 1691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "d5553e3d018c4cffadfee3b07d1dd138"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "ab83ae7fb18c9dbea61fba7ba5cfb1c1"
server_key_iv: "2de645266c46c3dea9c95efaaf551cc0"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.582935 1691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:48:20.583731 1706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.583913 1691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.584015 1691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "d5553e3d018c4cffadfee3b07d1dd138"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "ab83ae7fb18c9dbea61fba7ba5cfb1c1"
server_key_iv: "2de645266c46c3dea9c95efaaf551cc0"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.584087 1691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:20.598599 1691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:20.600694 1691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:20.600857 1691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:20.601097 1691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:20.601449 1691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:20.601509 1691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.601553 1691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:20.601598 1691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.607699 1691 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:33815
I20260620 18:48:20.607743 1819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:33815 every 8 connection(s)
I20260620 18:48:20.608099 1691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:48:20.609308 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 1691
I20260620 18:48:20.609417 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:48:20.609615 15669 external_mini_cluster.cc:1428] Setting key 81a984559ba6b7948c3590518fe59beb
Jun 20 18:48:20 dist-test-slave-tvl9 krb5kdc[1316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981300, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:20.617322 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.608885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:46643 (local address 127.15.77.126:34799)
0620 18:48:20.608947 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.608949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.609133 (+ 184us) server_negotiation.cc:407] Connection header received
0620 18:48:20.609735 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.609737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.609750 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.609810 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:20.610769 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.611269 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.612340 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.612544 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.612628 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.614680 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:20.614681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:20.614682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:20.614689 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:20.615966 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:20.616375 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:20.616376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:20.616376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:20.616432 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:20.616678 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:20.616679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:20.616680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:20.616796 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:20.616868 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.617148 (+ 280us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.617226 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 18:48:20.617537 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.608843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:34799 (local address 127.15.77.67:46643)
0620 18:48:20.609002 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:20.609044 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:20.609616 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:20.609881 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:20.609885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:20.610458 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:20.610663 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.610668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.611426 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.611427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.612228 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.612230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.612640 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.612641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.612766 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.613223 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:20.613232 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:20.614582 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:20.616064 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:20.616071 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:20.616073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:20.616291 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:20.616499 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:20.616500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:20.616501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:20.616608 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:20.616868 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:20.616877 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:20.617016 (+ 139us) client_negotiation.cc:769] Sending connection context
0620 18:48:20.617125 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 18:48:20.617286 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260620 18:48:20.618029 1820 heartbeater.cc:347] Connected to a master server at 127.15.77.126:34799
I20260620 18:48:20.618109 1820 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:20.618247 1820 heartbeater.cc:511] Master 127.15.77.126:34799 requested a full tablet report, sending...
I20260620 18:48:20.618621 1361 ts_manager.cc:194] Registered new tserver with Master: d5553e3d018c4cffadfee3b07d1dd138 (127.15.77.67:33815)
I20260620 18:48:20.619017 1361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:46643
I20260620 18:48:20.621093 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:48:20.628285 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.622387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36890 (local address 127.15.77.126:34799)
0620 18:48:20.622459 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.622461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.622564 (+ 103us) server_negotiation.cc:407] Connection header received
0620 18:48:20.622640 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.622641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.622651 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.622736 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:20.623370 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.623822 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.624494 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.624668 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.624717 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.625913 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:20.625915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:20.625915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:20.625921 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:20.627136 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:20.627550 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:20.627551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:20.627551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:20.627599 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:20.627764 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:20.627764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:20.627764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:20.627876 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:20.627989 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.628123 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.628170 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 18:48:20.629948 1361 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:48:20.631640 1361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:48:20.638525 1833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.634058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.15.77.66:35319)
0620 18:48:20.634816 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.634821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.634830 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:20.634865 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.634867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.634881 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.635106 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:20.635181 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.636188 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.637259 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.638031 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.638115 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.638180 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.638399 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.638444 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":79,"threads_started":1}
I20260620 18:48:20.638558 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.633963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:42983 (local address 127.0.0.1:53362)
0620 18:48:20.634219 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:20.634230 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:20.634283 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:20.634548 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:20.634551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:20.634553 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:20.634646 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.634649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.635977 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.635978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.637292 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.637295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.638177 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.638177 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.638270 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.638288 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 18:48:20.638374 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 18:48:20.638446 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":44,"thread_start_us":83,"threads_started":1}
I20260620 18:48:20.638557 1832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.633956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:35319 (local address 127.0.0.1:37918)
0620 18:48:20.634113 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:20.634152 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:20.634214 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:20.634990 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:20.634991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:20.634994 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:20.635034 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.635036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.636289 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.636290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.637173 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.637176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.638175 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.638177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.638270 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.638288 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 18:48:20.638374 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 18:48:20.638445 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":22,"thread_start_us":50,"threads_started":1}
I20260620 18:48:20.639204 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.634158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.15.77.65:42983)
0620 18:48:20.634352 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.634357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.634367 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:48:20.634413 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.634415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.634426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.634523 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:20.634730 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.635876 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.637386 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.638073 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.638138 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.638186 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.638832 (+ 646us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.638878 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
I20260620 18:48:20.639557 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.634347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.15.77.67:33815)
0620 18:48:20.634536 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.634542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.634700 (+ 158us) server_negotiation.cc:407] Connection header received
0620 18:48:20.634751 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.634753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.634769 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.634844 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:20.635025 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.635798 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.638223 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.638962 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.639039 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.639098 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.639437 (+ 339us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.639483 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260620 18:48:20.639285 1834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.634151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:33815 (local address 127.0.0.1:60698)
0620 18:48:20.634625 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:20.634635 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:20.634690 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:20.634912 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:20.634913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:20.634915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:20.634965 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.634967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.635906 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.635907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.637306 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.637308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.639049 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.639050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.639109 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.639115 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 18:48:20.639181 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 18:48:20.639221 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":69,"thread_start_us":66,"threads_started":1}
I20260620 18:48:20.639606 1620 tablet_service.cc:1467] Processing CreateTablet for tablet 26e538b22af04dc7b7194b97a74307d4 (DEFAULT_TABLE table=test-table [id=e9fda46343f14e5d945f1d7e6e8fa427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:20.640005 1620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26e538b22af04dc7b7194b97a74307d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:20.639954 1486 tablet_service.cc:1467] Processing CreateTablet for tablet 26e538b22af04dc7b7194b97a74307d4 (DEFAULT_TABLE table=test-table [id=e9fda46343f14e5d945f1d7e6e8fa427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:20.640251 1486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26e538b22af04dc7b7194b97a74307d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:20.642807 1837 tablet_bootstrap.cc:492] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: Bootstrap starting.
I20260620 18:48:20.643106 1838 tablet_bootstrap.cc:492] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b: Bootstrap starting.
I20260620 18:48:20.643478 1754 tablet_service.cc:1467] Processing CreateTablet for tablet 26e538b22af04dc7b7194b97a74307d4 (DEFAULT_TABLE table=test-table [id=e9fda46343f14e5d945f1d7e6e8fa427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:20.643625 1837 tablet_bootstrap.cc:654] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:20.643769 1838 tablet_bootstrap.cc:654] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:20.643800 1754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26e538b22af04dc7b7194b97a74307d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:20.644248 1837 log.cc:824] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:20.644347 1838 log.cc:824] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:20.645009 1837 tablet_bootstrap.cc:492] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: No bootstrap required, opened a new log
I20260620 18:48:20.645093 1837 ts_tablet_manager.cc:1397] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:48:20.645098 1838 tablet_bootstrap.cc:492] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b: No bootstrap required, opened a new log
I20260620 18:48:20.645162 1838 ts_tablet_manager.cc:1397] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 18:48:20.646642 1837 raft_consensus.cc:348] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 } } peers { permanent_uuid: "0a2eb2033c4749a8ba034ae4fece480b" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42983 } } peers { permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 } }
I20260620 18:48:20.646816 1837 raft_consensus.cc:374] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:20.646860 1837 raft_consensus.cc:729] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca653b78d675426e81045bbf2d276c4d, State: Initialized, Role: FOLLOWER
I20260620 18:48:20.646967 1837 consensus_queue.cc:260] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 } } peers { permanent_uuid: "0a2eb2033c4749a8ba034ae4fece480b" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42983 } } peers { permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 } }
I20260620 18:48:20.647248 1837 ts_tablet_manager.cc:1428] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:48:20.647229 1841 tablet_bootstrap.cc:492] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: Bootstrap starting.
I20260620 18:48:20.647253 1838 raft_consensus.cc:348] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 } } peers { permanent_uuid: "0a2eb2033c4749a8ba034ae4fece480b" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42983 } } peers { permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 } }
I20260620 18:48:20.647444 1838 raft_consensus.cc:374] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:20.647449 1686 heartbeater.cc:503] Master 127.15.77.126:34799 was elected leader, sending a full tablet report...
I20260620 18:48:20.647487 1838 raft_consensus.cc:729] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2eb2033c4749a8ba034ae4fece480b, State: Initialized, Role: FOLLOWER
I20260620 18:48:20.647622 1838 consensus_queue.cc:260] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 } } peers { permanent_uuid: "0a2eb2033c4749a8ba034ae4fece480b" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42983 } } peers { permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 } }
I20260620 18:48:20.647902 1838 ts_tablet_manager.cc:1428] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 18:48:20.647919 1552 heartbeater.cc:503] Master 127.15.77.126:34799 was elected leader, sending a full tablet report...
I20260620 18:48:20.647965 1841 tablet_bootstrap.cc:654] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:20.648535 1841 log.cc:824] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:20.649236 1841 tablet_bootstrap.cc:492] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: No bootstrap required, opened a new log
I20260620 18:48:20.649291 1841 ts_tablet_manager.cc:1397] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:48:20.650790 1841 raft_consensus.cc:348] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 } } peers { permanent_uuid: "0a2eb2033c4749a8ba034ae4fece480b" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42983 } } peers { permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 } }
I20260620 18:48:20.650924 1841 raft_consensus.cc:374] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:20.650947 1841 raft_consensus.cc:729] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5553e3d018c4cffadfee3b07d1dd138, State: Initialized, Role: FOLLOWER
I20260620 18:48:20.651084 1841 consensus_queue.cc:260] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 } } peers { permanent_uuid: "0a2eb2033c4749a8ba034ae4fece480b" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42983 } } peers { permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 } }
I20260620 18:48:20.651420 1820 heartbeater.cc:503] Master 127.15.77.126:34799 was elected leader, sending a full tablet report...
I20260620 18:48:20.651424 1841 ts_tablet_manager.cc:1428] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260620 18:48:20.702014 1687 tablet.cc:2369] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:48:20.749363 1843 raft_consensus.cc:482] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:20.749541 1843 raft_consensus.cc:504] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 } } peers { permanent_uuid: "0a2eb2033c4749a8ba034ae4fece480b" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42983 } } peers { permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 } }
I20260620 18:48:20.750054 1843 leader_election.cc:288] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca653b78d675426e81045bbf2d276c4d (127.15.77.66:35319), d5553e3d018c4cffadfee3b07d1dd138 (127.15.77.67:33815)
I20260620 18:48:20.753140 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.750288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:37561 (local address 127.15.77.67:33815)
0620 18:48:20.750341 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.750343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.750355 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:48:20.750453 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.750454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.750464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.750545 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:20.750710 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.751304 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.752371 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.752765 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.752858 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.752868 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.753033 (+ 165us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.753072 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 18:48:20.753149 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.750215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:33815 (local address 127.15.77.65:37561)
0620 18:48:20.750294 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:20.750307 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:20.750350 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:20.750561 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:20.750562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:20.750564 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:20.750639 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.750642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.751434 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.751436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.752265 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.752268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.752859 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.752859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.752944 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.752947 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:48:20.753024 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 18:48:20.753066 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":26}
I20260620 18:48:20.753142 1833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.750320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:40333 (local address 127.15.77.66:35319)
0620 18:48:20.750367 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.750369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.750637 (+ 268us) server_negotiation.cc:407] Connection header received
0620 18:48:20.750667 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.750668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.750683 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.750737 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:20.750903 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.751409 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.752382 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.752767 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.752857 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.752868 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.753034 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.753074 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 18:48:20.753159 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.750222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:35319 (local address 127.15.77.65:40333)
0620 18:48:20.750547 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:20.750558 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:20.750625 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:20.750766 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:20.750766 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:20.750768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:20.750831 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.750832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.751487 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.751488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.752278 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:20.752280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.752857 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.752858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:20.752932 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.752936 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:48:20.753015 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 18:48:20.753059 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":157,"thread_start_us":115,"threads_started":1}
I20260620 18:48:20.753547 1774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26e538b22af04dc7b7194b97a74307d4" candidate_uuid: "0a2eb2033c4749a8ba034ae4fece480b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5553e3d018c4cffadfee3b07d1dd138" is_pre_election: true
I20260620 18:48:20.753691 1774 raft_consensus.cc:2454] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2eb2033c4749a8ba034ae4fece480b in term 0.
I20260620 18:48:20.753744 1640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26e538b22af04dc7b7194b97a74307d4" candidate_uuid: "0a2eb2033c4749a8ba034ae4fece480b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca653b78d675426e81045bbf2d276c4d" is_pre_election: true
I20260620 18:48:20.753917 1640 raft_consensus.cc:2454] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a2eb2033c4749a8ba034ae4fece480b in term 0.
I20260620 18:48:20.753916 1440 leader_election.cc:302] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2eb2033c4749a8ba034ae4fece480b, d5553e3d018c4cffadfee3b07d1dd138; no voters:
I20260620 18:48:20.754051 1843 raft_consensus.cc:2781] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:48:20.754108 1843 raft_consensus.cc:482] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:48:20.754139 1843 raft_consensus.cc:3037] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:20.754865 1843 raft_consensus.cc:504] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 } } peers { permanent_uuid: "0a2eb2033c4749a8ba034ae4fece480b" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42983 } } peers { permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 } }
I20260620 18:48:20.755017 1843 leader_election.cc:288] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [CANDIDATE]: Term 1 election: Requested vote from peers ca653b78d675426e81045bbf2d276c4d (127.15.77.66:35319), d5553e3d018c4cffadfee3b07d1dd138 (127.15.77.67:33815)
I20260620 18:48:20.755168 1774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26e538b22af04dc7b7194b97a74307d4" candidate_uuid: "0a2eb2033c4749a8ba034ae4fece480b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5553e3d018c4cffadfee3b07d1dd138"
I20260620 18:48:20.755191 1640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26e538b22af04dc7b7194b97a74307d4" candidate_uuid: "0a2eb2033c4749a8ba034ae4fece480b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca653b78d675426e81045bbf2d276c4d"
I20260620 18:48:20.755259 1774 raft_consensus.cc:3037] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:20.755282 1640 raft_consensus.cc:3037] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:20.756017 1640 raft_consensus.cc:2454] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2eb2033c4749a8ba034ae4fece480b in term 1.
I20260620 18:48:20.756044 1774 raft_consensus.cc:2454] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a2eb2033c4749a8ba034ae4fece480b in term 1.
I20260620 18:48:20.756199 1441 leader_election.cc:302] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a2eb2033c4749a8ba034ae4fece480b, ca653b78d675426e81045bbf2d276c4d; no voters:
I20260620 18:48:20.756304 1843 raft_consensus.cc:2781] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:20.756440 1843 raft_consensus.cc:686] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [term 1 LEADER]: Becoming Leader. State: Replica: 0a2eb2033c4749a8ba034ae4fece480b, State: Running, Role: LEADER
I20260620 18:48:20.756549 1843 consensus_queue.cc:237] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 } } peers { permanent_uuid: "0a2eb2033c4749a8ba034ae4fece480b" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42983 } } peers { permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 } }
I20260620 18:48:20.757345 1361 catalog_manager.cc:5495] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a2eb2033c4749a8ba034ae4fece480b (127.15.77.65). New cstate: current_term: 1 leader_uuid: "0a2eb2033c4749a8ba034ae4fece480b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a2eb2033c4749a8ba034ae4fece480b" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 42983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 } health_report { overall_health: UNKNOWN } } }
I20260620 18:48:20.783671 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.781366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.15.77.65:42983)
0620 18:48:20.781429 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.781431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.781438 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:48:20.781476 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.781477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.781499 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.781566 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:20.781683 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.782154 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.782904 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.783102 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.783156 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.783282 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:20.783440 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:20.783524 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.783577 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.783588 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 18:48:20.785588 1640 raft_consensus.cc:1261] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d [term 1 FOLLOWER]: Refusing update from remote peer 0a2eb2033c4749a8ba034ae4fece480b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:20.785590 1774 raft_consensus.cc:1261] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138 [term 1 FOLLOWER]: Refusing update from remote peer 0a2eb2033c4749a8ba034ae4fece480b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:20.785997 1847 consensus_queue.cc:1035] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca653b78d675426e81045bbf2d276c4d" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 35319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:20.786146 1843 consensus_queue.cc:1035] T 26e538b22af04dc7b7194b97a74307d4 P 0a2eb2033c4749a8ba034ae4fece480b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5553e3d018c4cffadfee3b07d1dd138" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:20.788682 1850 mvcc.cc:205] Tried to move back new op lower bound from 7298995408014983168 to 7298995407899451392. Current Snapshot: MvccSnapshot[applied={T|T < 7298995408014983168}]
I20260620 18:48:20.788910 1851 mvcc.cc:205] Tried to move back new op lower bound from 7298995408014983168 to 7298995407899451392. Current Snapshot: MvccSnapshot[applied={T|T < 7298995408014983168}]
I20260620 18:48:20.790586 1852 mvcc.cc:205] Tried to move back new op lower bound from 7298995408014983168 to 7298995407899451392. Current Snapshot: MvccSnapshot[applied={T|T < 7298995408014983168}]
I20260620 18:48:20.791692 1361 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:20.791798 1361 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:20.792526 1361 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8812a83973b34d9d8f81184523f62e44: Sending DeleteTablet for 3 replicas of tablet 26e538b22af04dc7b7194b97a74307d4
I20260620 18:48:20.792939 1754 tablet_service.cc:1514] Processing DeleteTablet for tablet 26e538b22af04dc7b7194b97a74307d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:20 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:60698
I20260620 18:48:20.792914 1620 tablet_service.cc:1514] Processing DeleteTablet for tablet 26e538b22af04dc7b7194b97a74307d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:20 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:37918
I20260620 18:48:20.793231 1861 tablet_replica.cc:330] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: stopping tablet replica
I20260620 18:48:20.793211 1486 tablet_service.cc:1514] Processing DeleteTablet for tablet 26e538b22af04dc7b7194b97a74307d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:20 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:53362
I20260620 18:48:20.793331 1861 raft_consensus.cc:2229] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:48:20.793401 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 1423
I20260620 18:48:20.795632 1860 tablet_replica.cc:330] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: stopping tablet replica
I20260620 18:48:20.795797 1860 raft_consensus.cc:2229] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:48:20.795948 1860 raft_consensus.cc:2258] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:20.796147 1861 raft_consensus.cc:2258] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:20.796660 1860 ts_tablet_manager.cc:1905] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:20.796660 1861 ts_tablet_manager.cc:1905] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:20.798907 1860 ts_tablet_manager.cc:1918] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:20.798916 1861 ts_tablet_manager.cc:1918] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:20.799000 1860 log.cc:1196] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/26e538b22af04dc7b7194b97a74307d4
I20260620 18:48:20.799074 1861 log.cc:1196] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/26e538b22af04dc7b7194b97a74307d4
I20260620 18:48:20.799347 1860 ts_tablet_manager.cc:1939] T 26e538b22af04dc7b7194b97a74307d4 P d5553e3d018c4cffadfee3b07d1dd138: Deleting consensus metadata
I20260620 18:48:20.799468 1861 ts_tablet_manager.cc:1939] T 26e538b22af04dc7b7194b97a74307d4 P ca653b78d675426e81045bbf2d276c4d: Deleting consensus metadata
I20260620 18:48:20.799942 1346 catalog_manager.cc:4841] TS d5553e3d018c4cffadfee3b07d1dd138 (127.15.77.67:33815): tablet 26e538b22af04dc7b7194b97a74307d4 (table test-table [id=e9fda46343f14e5d945f1d7e6e8fa427]) successfully deleted
I20260620 18:48:20.800139 1347 catalog_manager.cc:4841] TS ca653b78d675426e81045bbf2d276c4d (127.15.77.66:35319): tablet 26e538b22af04dc7b7194b97a74307d4 (table test-table [id=e9fda46343f14e5d945f1d7e6e8fa427]) successfully deleted
W20260620 18:48:20.804116 1346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:42983 (error 108)
I20260620 18:48:20.804177 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 1557
I20260620 18:48:20.805115 1834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.804977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:42983 (local address 127.0.0.1:53386)
0620 18:48:20.805033 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:20.805071 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 18:48:20.805239 1346 catalog_manager.cc:4584] TS 0a2eb2033c4749a8ba034ae4fece480b (127.15.77.65:42983): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26e538b22af04dc7b7194b97a74307d4: Network error: Client connection negotiation failed: client connection to 127.15.77.65:42983: connect: Connection refused (error 111)
I20260620 18:48:20.809541 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 1691
I20260620 18:48:20.815191 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 1329
2026-06-20T18:48:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2000 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-20T18:48:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:20Z Disabled control of system clock
I20260620 18:48:20.843843 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:39065
--webserver_interface=127.15.77.126
--webserver_port=0
--builtin_ntp_servers=127.15.77.84:39759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:39065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 18:48:20.927197 1867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:20.927477 1867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:20.927500 1867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:20.928754 1867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:20.928804 1867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:20.928817 1867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:20.928830 1867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:20.928843 1867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:20.930190 1867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:39759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:39065
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:39065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:20.930421 1867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:20.930670 1867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:20.935678 1872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:20.935702 1875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:20.935693 1873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:20.935846 1867 server_base.cc:1034] running on GCE node
I20260620 18:48:20.936234 1867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:20.936589 1867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:20.937747 1867 hybrid_clock.cc:648] HybridClock initialized: now 1781981300937728 us; error 38 us; skew 500 ppm
I20260620 18:48:20.939200 1867 webserver.cc:460] Webserver started at http://127.15.77.126:39637/ using document root <none> and password file <none>
I20260620 18:48:20.939522 1867 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:20.939599 1867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:20.939754 1867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:20.940860 1867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "b25e7607f29040bb85de8f144846f574"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "787c8b36c281d5695d9e57f71f68a731"
server_key_iv: "73a0d17107de041eaf6343f52bd19d02"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.941226 1867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "b25e7607f29040bb85de8f144846f574"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "787c8b36c281d5695d9e57f71f68a731"
server_key_iv: "73a0d17107de041eaf6343f52bd19d02"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.942538 1867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:20.943279 1881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:20.943483 1867 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:20.943578 1867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "b25e7607f29040bb85de8f144846f574"
format_stamp: "Formatted at 2026-06-20 18:48:20 on dist-test-slave-tvl9"
server_key: "787c8b36c281d5695d9e57f71f68a731"
server_key_iv: "73a0d17107de041eaf6343f52bd19d02"
server_key_version: "encryptionkey@0"
I20260620 18:48:20.943646 1867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:20.967459 1867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:20.967763 1867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:20.967868 1867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:20.971800 1867 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:39065
I20260620 18:48:20.971854 1933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:39065 every 8 connection(s)
I20260620 18:48:20.972219 1867 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:20.973306 1934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:20.975725 1934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574: Bootstrap starting.
I20260620 18:48:20.976285 1934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:20.976540 1934 log.cc:824] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:20.977248 1934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574: No bootstrap required, opened a new log
I20260620 18:48:20.978979 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 1867
I20260620 18:48:20.979082 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:20.978979 1934 raft_consensus.cc:348] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25e7607f29040bb85de8f144846f574" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39065 } }
I20260620 18:48:20.979173 1934 raft_consensus.cc:374] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:20.979211 1934 raft_consensus.cc:729] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b25e7607f29040bb85de8f144846f574, State: Initialized, Role: FOLLOWER
I20260620 18:48:20.979269 15669 external_mini_cluster.cc:1428] Setting key 5256a11ce8abff4377b47ddd35428d1b
I20260620 18:48:20.979414 1934 consensus_queue.cc:260] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25e7607f29040bb85de8f144846f574" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39065 } }
I20260620 18:48:20.979511 1934 raft_consensus.cc:388] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:20.979571 1934 raft_consensus.cc:482] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:20.979624 1934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:20.980345 1934 raft_consensus.cc:504] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25e7607f29040bb85de8f144846f574" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39065 } }
I20260620 18:48:20.980466 1934 leader_election.cc:302] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b25e7607f29040bb85de8f144846f574; no voters:
I20260620 18:48:20.980647 1934 leader_election.cc:288] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:20.980671 1939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:20.980819 1939 raft_consensus.cc:686] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [term 1 LEADER]: Becoming Leader. State: Replica: b25e7607f29040bb85de8f144846f574, State: Running, Role: LEADER
I20260620 18:48:20.980906 1939 consensus_queue.cc:237] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25e7607f29040bb85de8f144846f574" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39065 } }
I20260620 18:48:20.980924 1934 sys_catalog.cc:564] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:20.981249 1939 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b25e7607f29040bb85de8f144846f574. Latest consensus state: current_term: 1 leader_uuid: "b25e7607f29040bb85de8f144846f574" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25e7607f29040bb85de8f144846f574" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39065 } } }
I20260620 18:48:20.981353 1939 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:20.981266 1940 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b25e7607f29040bb85de8f144846f574" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25e7607f29040bb85de8f144846f574" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 39065 } } }
I20260620 18:48:20.981564 1940 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:20.981678 1943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:20.982143 1943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:20.983790 1943 catalog_manager.cc:1347] Generated new cluster ID: 6471cd5e23b0414095a0446196401698
I20260620 18:48:20.983843 1943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:20.985888 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:20.979880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37374 (local address 127.15.77.126:39065)
0620 18:48:20.980132 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:20.980145 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:20.980171 (+ 26us) server_negotiation.cc:407] Connection header received
0620 18:48:20.980271 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:20.980277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:20.980339 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:20.980503 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:20.980753 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.982522 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.983632 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:20.984249 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:20.984371 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:20.985162 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:20.985166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:20.985166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:20.985376 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:20.985417 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:20.985431 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:20.985505 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:20.985587 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 18:48:20.985676 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":131,"threads_started":1}
I20260620 18:48:21.003880 1943 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:21.004740 1943 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:21.011890 1943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574: Generated new TSK 0
I20260620 18:48:21.012122 1943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:21.014158 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39065
--builtin_ntp_servers=127.15.77.84:39759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 18:48:21.100090 1958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:21.100304 1958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:21.100327 1958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:21.101524 1958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:21.101567 1958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:21.101593 1958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:48:21.103070 1958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:39759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:21.103374 1958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:21.103716 1958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:21.109776 1966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:21.109812 1964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:21.109877 1958 server_base.cc:1034] running on GCE node
W20260620 18:48:21.109822 1963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:21.110317 1958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:21.110610 1958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:21.111776 1958 hybrid_clock.cc:648] HybridClock initialized: now 1781981301111764 us; error 29 us; skew 500 ppm
I20260620 18:48:21.116990 1958 webserver.cc:460] Webserver started at http://127.15.77.65:39211/ using document root <none> and password file <none>
I20260620 18:48:21.117327 1958 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:21.117430 1958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:21.117600 1958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:21.118791 1958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "d3575ee3ea5c47f9906608f79a353c2e"
format_stamp: "Formatted at 2026-06-20 18:48:21 on dist-test-slave-tvl9"
server_key: "934eab9cc9a3d15bd80965475b960db4"
server_key_iv: "336e468511484456d0c9724bf50ab4ec"
server_key_version: "encryptionkey@0"
I20260620 18:48:21.119167 1958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "d3575ee3ea5c47f9906608f79a353c2e"
format_stamp: "Formatted at 2026-06-20 18:48:21 on dist-test-slave-tvl9"
server_key: "934eab9cc9a3d15bd80965475b960db4"
server_key_iv: "336e468511484456d0c9724bf50ab4ec"
server_key_version: "encryptionkey@0"
I20260620 18:48:21.120738 1958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:21.121555 1972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:21.121805 1958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:21.121915 1958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "d3575ee3ea5c47f9906608f79a353c2e"
format_stamp: "Formatted at 2026-06-20 18:48:21 on dist-test-slave-tvl9"
server_key: "934eab9cc9a3d15bd80965475b960db4"
server_key_iv: "336e468511484456d0c9724bf50ab4ec"
server_key_version: "encryptionkey@0"
I20260620 18:48:21.122006 1958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:21.133210 1958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:21.133558 1958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:21.133723 1958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:21.133957 1958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:21.134305 1958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:21.134361 1958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:21.134470 1958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:21.134502 1958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:21.140867 1958 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:40507
I20260620 18:48:21.140918 2085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:40507 every 8 connection(s)
I20260620 18:48:21.141254 1958 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:48:21.146540 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.141985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:39271 (local address 127.15.77.126:39065)
0620 18:48:21.142064 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:21.142067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:21.142319 (+ 252us) server_negotiation.cc:407] Connection header received
0620 18:48:21.142823 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:21.142824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:21.142837 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:21.142895 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:21.143678 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.144182 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.145130 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.145324 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.145388 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.146288 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:21.146289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:21.146289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:21.146293 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:21.146306 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:21.146309 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:21.146350 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:21.146443 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 18:48:21.146477 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 18:48:21.146780 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.141988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:39065 (local address 127.15.77.65:39271)
0620 18:48:21.142206 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:21.142242 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:21.142729 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:21.142981 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:21.142985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:21.142993 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:21.143593 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.143599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.144268 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.144269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.145054 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.145055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.145401 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.145402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.145511 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.146129 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:48:21.146130 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:21.146159 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:48:21.146197 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:21.146370 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:21.146375 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:21.146378 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:48:21.146435 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 18:48:21.146527 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260620 18:48:21.147374 2086 heartbeater.cc:347] Connected to a master server at 127.15.77.126:39065
I20260620 18:48:21.147493 2086 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:21.147680 2086 heartbeater.cc:511] Master 127.15.77.126:39065 requested a full tablet report, sending...
I20260620 18:48:21.148204 1898 ts_manager.cc:194] Registered new tserver with Master: d3575ee3ea5c47f9906608f79a353c2e (127.15.77.65:40507)
I20260620 18:48:21.148855 1898 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.65:39271
I20260620 18:48:21.149433 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 1958
I20260620 18:48:21.149521 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:48:21.149734 15669 external_mini_cluster.cc:1428] Setting key b96481b6e389fb71f2234f6d71bc279e
I20260620 18:48:21.150722 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39065
--builtin_ntp_servers=127.15.77.84:39759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 18:48:21.235836 2089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:21.236030 2089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:21.236052 2089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:21.237212 2089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:21.237252 2089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:21.237278 2089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:48:21.238627 2089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:39759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:21.238863 2089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:21.239159 2089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:21.244100 2097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:21.244112 2095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:21.244315 2089 server_base.cc:1034] running on GCE node
W20260620 18:48:21.244426 2094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:21.244674 2089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:21.244870 2089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:21.245985 2089 hybrid_clock.cc:648] HybridClock initialized: now 1781981301245975 us; error 37 us; skew 500 ppm
I20260620 18:48:21.247260 2089 webserver.cc:460] Webserver started at http://127.15.77.66:43219/ using document root <none> and password file <none>
I20260620 18:48:21.247545 2089 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:21.247594 2089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:21.247735 2089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:21.248792 2089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "0a21a4d7cc9f4021922f4262f4db5bb1"
format_stamp: "Formatted at 2026-06-20 18:48:21 on dist-test-slave-tvl9"
server_key: "b7b91f9156605fadbbde04d055967ccd"
server_key_iv: "28abc7480c2d36bca3f88f87d1b71a8d"
server_key_version: "encryptionkey@0"
I20260620 18:48:21.249142 2089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "0a21a4d7cc9f4021922f4262f4db5bb1"
format_stamp: "Formatted at 2026-06-20 18:48:21 on dist-test-slave-tvl9"
server_key: "b7b91f9156605fadbbde04d055967ccd"
server_key_iv: "28abc7480c2d36bca3f88f87d1b71a8d"
server_key_version: "encryptionkey@0"
I20260620 18:48:21.250424 2089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:21.251127 2103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:21.251291 2089 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:21.251420 2089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "0a21a4d7cc9f4021922f4262f4db5bb1"
format_stamp: "Formatted at 2026-06-20 18:48:21 on dist-test-slave-tvl9"
server_key: "b7b91f9156605fadbbde04d055967ccd"
server_key_iv: "28abc7480c2d36bca3f88f87d1b71a8d"
server_key_version: "encryptionkey@0"
I20260620 18:48:21.251505 2089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:21.270624 2089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:21.270964 2089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:21.271123 2089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:21.271384 2089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:21.271742 2089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:21.271797 2089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:21.271838 2089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:21.271867 2089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:21.277650 2089 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:44291
I20260620 18:48:21.277690 2216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:44291 every 8 connection(s)
I20260620 18:48:21.278031 2089 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:48:21.283506 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.278830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:54607 (local address 127.15.77.126:39065)
0620 18:48:21.278880 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:21.278882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:21.279150 (+ 268us) server_negotiation.cc:407] Connection header received
0620 18:48:21.279666 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:21.279667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:21.279677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:21.279732 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:21.280396 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.280897 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.281948 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.282143 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.282259 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.283131 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:21.283133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:21.283133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:21.283138 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:21.283153 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:21.283157 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:21.283229 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:21.283350 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 18:48:21.283424 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:48:21.283692 2219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.278828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:39065 (local address 127.15.77.66:54607)
0620 18:48:21.279018 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:21.279055 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:21.279581 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:21.279786 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:21.279789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:21.279798 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:21.280323 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.280328 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.280990 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.280991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.281866 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.281869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.282300 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.282301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.282451 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.282982 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:48:21.282982 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:21.283012 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:48:21.283036 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:21.283230 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:21.283237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:21.283240 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:48:21.283350 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 18:48:21.283459 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260620 18:48:21.284204 2217 heartbeater.cc:347] Connected to a master server at 127.15.77.126:39065
I20260620 18:48:21.284300 2217 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:21.284444 2217 heartbeater.cc:511] Master 127.15.77.126:39065 requested a full tablet report, sending...
I20260620 18:48:21.284822 1898 ts_manager.cc:194] Registered new tserver with Master: 0a21a4d7cc9f4021922f4262f4db5bb1 (127.15.77.66:44291)
I20260620 18:48:21.285200 1898 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.66:54607
I20260620 18:48:21.285701 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 2089
I20260620 18:48:21.285807 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:48:21.285989 15669 external_mini_cluster.cc:1428] Setting key 9d9335bb7c4a758791f42efa7fbc56e7
I20260620 18:48:21.286909 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39065
--builtin_ntp_servers=127.15.77.84:39759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 18:48:21.368798 2220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:21.369051 2220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:21.369087 2220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:21.370275 2220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:21.370344 2220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:21.370385 2220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:48:21.371763 2220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:39759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:39065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:21.372041 2220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:21.372342 2220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:21.380885 2225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:21.380911 2226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:21.381516 2220 server_base.cc:1034] running on GCE node
W20260620 18:48:21.381697 2228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:21.381966 2220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:21.382227 2220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:21.383390 2220 hybrid_clock.cc:648] HybridClock initialized: now 1781981301383371 us; error 33 us; skew 500 ppm
I20260620 18:48:21.384663 2220 webserver.cc:460] Webserver started at http://127.15.77.67:44295/ using document root <none> and password file <none>
I20260620 18:48:21.384922 2220 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:21.384968 2220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:21.385135 2220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:21.386258 2220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "ed70995f6ae54349b228842cc541b68d"
format_stamp: "Formatted at 2026-06-20 18:48:21 on dist-test-slave-tvl9"
server_key: "1241721ba4aeba0b63b4a8537026096a"
server_key_iv: "6ad0d0394d566c679172088328a04558"
server_key_version: "encryptionkey@0"
I20260620 18:48:21.386613 2220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "ed70995f6ae54349b228842cc541b68d"
format_stamp: "Formatted at 2026-06-20 18:48:21 on dist-test-slave-tvl9"
server_key: "1241721ba4aeba0b63b4a8537026096a"
server_key_iv: "6ad0d0394d566c679172088328a04558"
server_key_version: "encryptionkey@0"
I20260620 18:48:21.388000 2220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:21.388698 2234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:21.388859 2220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:21.388974 2220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "ed70995f6ae54349b228842cc541b68d"
format_stamp: "Formatted at 2026-06-20 18:48:21 on dist-test-slave-tvl9"
server_key: "1241721ba4aeba0b63b4a8537026096a"
server_key_iv: "6ad0d0394d566c679172088328a04558"
server_key_version: "encryptionkey@0"
I20260620 18:48:21.389052 2220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:21.397559 2220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:21.397868 2220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:21.398022 2220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:21.398253 2220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:21.398578 2220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:21.398633 2220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:21.398675 2220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:21.398702 2220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:21.404485 2220 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:46421
I20260620 18:48:21.404567 2347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:46421 every 8 connection(s)
I20260620 18:48:21.404927 2220 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:48:21.410619 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.405849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:55477 (local address 127.15.77.126:39065)
0620 18:48:21.405935 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:21.405937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:21.406235 (+ 298us) server_negotiation.cc:407] Connection header received
0620 18:48:21.406733 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:21.406734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:21.406746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:21.406820 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:21.407630 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.408146 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.409136 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.409323 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.409380 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.410344 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:21.410345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:21.410345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:21.410364 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:21.410378 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:21.410382 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:21.410426 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:21.410524 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 18:48:21.410552 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 18:48:21.410800 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.405825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:39065 (local address 127.15.77.67:55477)
0620 18:48:21.406107 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:21.406144 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:21.406626 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:21.406898 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:21.406902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:21.406922 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:21.407540 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.407545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.408230 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.408231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.409053 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.409055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.409401 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.409401 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.409512 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.410158 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:48:21.410159 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:21.410222 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:48:21.410253 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:21.410444 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:21.410451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:21.410457 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 18:48:21.410513 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 18:48:21.410594 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":105,"threads_started":1}
I20260620 18:48:21.411263 2348 heartbeater.cc:347] Connected to a master server at 127.15.77.126:39065
I20260620 18:48:21.411410 2348 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:21.411653 2348 heartbeater.cc:511] Master 127.15.77.126:39065 requested a full tablet report, sending...
I20260620 18:48:21.411823 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 2220
I20260620 18:48:21.411942 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:48:21.412091 1898 ts_manager.cc:194] Registered new tserver with Master: ed70995f6ae54349b228842cc541b68d (127.15.77.67:46421)
I20260620 18:48:21.412145 15669 external_mini_cluster.cc:1428] Setting key 386b58318e849021499e82795a0c2340
I20260620 18:48:21.412541 1898 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.67:55477
I20260620 18:48:21.413094 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:48:21.417266 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.414559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35136 (local address 127.15.77.126:39065)
0620 18:48:21.414624 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:21.414626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:21.414752 (+ 126us) server_negotiation.cc:407] Connection header received
0620 18:48:21.414779 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:21.414780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:21.414790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:21.414847 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:21.414963 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.415506 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.416220 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.416395 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.416446 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.417053 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:21.417054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:21.417054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:21.417059 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:21.417071 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:21.417075 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:21.417145 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:21.417203 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 18:48:21.417212 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 18:48:21.418852 1898 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:48:21.420580 1898 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:48:21.427085 2356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.422736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:46421 (local address 127.0.0.1:49754)
0620 18:48:21.422872 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:21.422911 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:21.422973 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:21.423347 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:21.423350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:21.423352 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:21.423417 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.423420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.424745 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.424746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.426074 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.426077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.426770 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.426771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.426848 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.426862 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 18:48:21.426948 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 18:48:21.427003 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":77,"thread_start_us":81,"threads_started":1}
I20260620 18:48:21.427096 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.422855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.15.77.67:46421)
0620 18:48:21.423125 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:21.423130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:21.423140 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:48:21.423183 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:21.423186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:21.423206 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:21.423307 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:21.423505 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.424658 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.426170 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.426670 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.426770 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.426870 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:21.426964 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 18:48:21.427001 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":114,"threads_started":1}
I20260620 18:48:21.427225 2358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.422975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:40507 (local address 127.0.0.1:39316)
0620 18:48:21.423108 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:21.423120 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:21.423178 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:21.423725 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:21.423726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:21.423728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:21.423766 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.423768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.425026 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.425027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.426093 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.426096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.427017 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.427017 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.427077 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.427081 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:48:21.427110 (+ 29us) client_negotiation.cc:240] Negotiation successful
0620 18:48:21.427152 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260620 18:48:21.427783 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.423072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39316 (local address 127.15.77.65:40507)
0620 18:48:21.423586 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:21.423590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:21.423600 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:48:21.423637 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:21.423639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:21.423653 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:21.423729 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:21.423859 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.424904 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.426190 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.426901 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.427533 (+ 632us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.427637 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:21.427664 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 18:48:21.427702 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":88,"threads_started":1}
I20260620 18:48:21.427903 2282 tablet_service.cc:1467] Processing CreateTablet for tablet c3804433eb5a40d38057f973b38af0cd (DEFAULT_TABLE table=test-table [id=27b86ce564904f4ead1b465bf9a04321]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:21.428308 2282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3804433eb5a40d38057f973b38af0cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:21.428437 2360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.423151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:44291 (local address 127.0.0.1:48918)
0620 18:48:21.423277 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:21.423288 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:21.423370 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:21.423836 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:21.423837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:21.423839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:21.423888 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.423889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.424969 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.424970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.426970 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.426973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.428231 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.428232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.428295 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.428300 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:48:21.428330 (+ 30us) client_negotiation.cc:240] Negotiation successful
0620 18:48:21.428370 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":836,"thread_start_us":78,"threads_started":1}
I20260620 18:48:21.428611 2020 tablet_service.cc:1467] Processing CreateTablet for tablet c3804433eb5a40d38057f973b38af0cd (DEFAULT_TABLE table=test-table [id=27b86ce564904f4ead1b465bf9a04321]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:21.428805 2361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.423380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48918 (local address 127.15.77.66:44291)
0620 18:48:21.423525 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:21.423530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:21.423541 (+ 11us) server_negotiation.cc:407] Connection header received
0620 18:48:21.423582 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:21.423585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:21.423600 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:21.423967 (+ 367us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:21.424016 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.424870 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.427059 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.428135 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.428559 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.428658 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:21.428685 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 18:48:21.428725 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260620 18:48:21.428944 2020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3804433eb5a40d38057f973b38af0cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:21.429467 2151 tablet_service.cc:1467] Processing CreateTablet for tablet c3804433eb5a40d38057f973b38af0cd (DEFAULT_TABLE table=test-table [id=27b86ce564904f4ead1b465bf9a04321]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:21.429816 2151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3804433eb5a40d38057f973b38af0cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:21.431382 2362 tablet_bootstrap.cc:492] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e: Bootstrap starting.
I20260620 18:48:21.432096 2362 tablet_bootstrap.cc:654] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:21.432372 2362 log.cc:824] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:21.432751 2363 tablet_bootstrap.cc:492] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: Bootstrap starting.
I20260620 18:48:21.432956 2365 tablet_bootstrap.cc:492] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: Bootstrap starting.
I20260620 18:48:21.433228 2362 tablet_bootstrap.cc:492] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e: No bootstrap required, opened a new log
I20260620 18:48:21.433282 2362 ts_tablet_manager.cc:1397] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:48:21.433581 2365 tablet_bootstrap.cc:654] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:21.433873 2365 log.cc:824] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:21.435479 2362 raft_consensus.cc:348] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed70995f6ae54349b228842cc541b68d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 46421 } } peers { permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 } } peers { permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 } }
I20260620 18:48:21.435678 2362 raft_consensus.cc:374] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:21.435715 2362 raft_consensus.cc:729] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3575ee3ea5c47f9906608f79a353c2e, State: Initialized, Role: FOLLOWER
I20260620 18:48:21.435828 2362 consensus_queue.cc:260] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed70995f6ae54349b228842cc541b68d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 46421 } } peers { permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 } } peers { permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 } }
I20260620 18:48:21.435990 2363 tablet_bootstrap.cc:654] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:21.436116 2362 ts_tablet_manager.cc:1428] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:48:21.436280 2086 heartbeater.cc:503] Master 127.15.77.126:39065 was elected leader, sending a full tablet report...
I20260620 18:48:21.436314 2363 log.cc:824] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:21.436896 2365 tablet_bootstrap.cc:492] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: No bootstrap required, opened a new log
I20260620 18:48:21.436969 2365 ts_tablet_manager.cc:1397] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 18:48:21.437084 2363 tablet_bootstrap.cc:492] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: No bootstrap required, opened a new log
I20260620 18:48:21.437157 2363 ts_tablet_manager.cc:1397] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 18:48:21.439157 2365 raft_consensus.cc:348] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed70995f6ae54349b228842cc541b68d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 46421 } } peers { permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 } } peers { permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 } }
I20260620 18:48:21.439283 2365 raft_consensus.cc:374] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:21.439306 2365 raft_consensus.cc:729] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a21a4d7cc9f4021922f4262f4db5bb1, State: Initialized, Role: FOLLOWER
I20260620 18:48:21.439316 2363 raft_consensus.cc:348] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed70995f6ae54349b228842cc541b68d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 46421 } } peers { permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 } } peers { permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 } }
I20260620 18:48:21.439529 2363 raft_consensus.cc:374] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:21.439513 2365 consensus_queue.cc:260] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed70995f6ae54349b228842cc541b68d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 46421 } } peers { permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 } } peers { permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 } }
I20260620 18:48:21.439574 2363 raft_consensus.cc:729] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed70995f6ae54349b228842cc541b68d, State: Initialized, Role: FOLLOWER
I20260620 18:48:21.439689 2363 consensus_queue.cc:260] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed70995f6ae54349b228842cc541b68d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 46421 } } peers { permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 } } peers { permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 } }
I20260620 18:48:21.439762 2365 ts_tablet_manager.cc:1428] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 18:48:21.439895 2217 heartbeater.cc:503] Master 127.15.77.126:39065 was elected leader, sending a full tablet report...
I20260620 18:48:21.439967 2363 ts_tablet_manager.cc:1428] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 18:48:21.440044 2348 heartbeater.cc:503] Master 127.15.77.126:39065 was elected leader, sending a full tablet report...
I20260620 18:48:21.527143 2370 raft_consensus.cc:482] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:21.527274 2370 raft_consensus.cc:504] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed70995f6ae54349b228842cc541b68d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 46421 } } peers { permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 } } peers { permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 } }
I20260620 18:48:21.527706 2370 leader_election.cc:288] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3575ee3ea5c47f9906608f79a353c2e (127.15.77.65:40507), 0a21a4d7cc9f4021922f4262f4db5bb1 (127.15.77.66:44291)
W20260620 18:48:21.528618 2218 tablet.cc:2369] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:48:21.530831 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.528015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:57849 (local address 127.15.77.65:40507)
0620 18:48:21.528066 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:21.528068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:21.528080 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:48:21.528145 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:21.528146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:21.528158 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:21.528223 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:21.528437 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.529096 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.530049 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.530448 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.530505 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.530513 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:21.530704 (+ 191us) server_negotiation.cc:299] Negotiation successful
0620 18:48:21.530750 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 18:48:21.530848 2361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.528116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:56225 (local address 127.15.77.66:44291)
0620 18:48:21.528185 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:21.528187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:21.528292 (+ 105us) server_negotiation.cc:407] Connection header received
0620 18:48:21.528344 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:21.528345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:21.528356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:21.528413 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:21.528631 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.529199 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.530136 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.530469 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.530563 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.530571 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:21.530744 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 18:48:21.530784 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 18:48:21.530840 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.527906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:40507 (local address 127.15.77.67:57849)
0620 18:48:21.527972 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:21.527986 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:21.528073 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:21.528236 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:21.528237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:21.528240 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:21.528338 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.528340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.529177 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.529178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.529960 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.529962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.530549 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.530551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.530622 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.530625 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:48:21.530678 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 18:48:21.530739 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 18:48:21.530848 2371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.528045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:44291 (local address 127.15.77.67:56225)
0620 18:48:21.528219 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:21.528230 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:21.528286 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:21.528433 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:21.528434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:21.528436 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:21.528544 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.528546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.529285 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.529286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.530043 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:21.530045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.530583 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.530584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:21.530647 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.530649 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:48:21.530724 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 18:48:21.530759 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260620 18:48:21.531208 2040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3804433eb5a40d38057f973b38af0cd" candidate_uuid: "ed70995f6ae54349b228842cc541b68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3575ee3ea5c47f9906608f79a353c2e" is_pre_election: true
I20260620 18:48:21.531401 2040 raft_consensus.cc:2454] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed70995f6ae54349b228842cc541b68d in term 0.
I20260620 18:48:21.531706 2238 leader_election.cc:302] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3575ee3ea5c47f9906608f79a353c2e, ed70995f6ae54349b228842cc541b68d; no voters:
I20260620 18:48:21.531837 2370 raft_consensus.cc:2781] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:48:21.531817 2171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3804433eb5a40d38057f973b38af0cd" candidate_uuid: "ed70995f6ae54349b228842cc541b68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" is_pre_election: true
I20260620 18:48:21.531895 2370 raft_consensus.cc:482] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:48:21.531930 2370 raft_consensus.cc:3037] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:21.531949 2171 raft_consensus.cc:2454] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed70995f6ae54349b228842cc541b68d in term 0.
I20260620 18:48:21.532686 2370 raft_consensus.cc:504] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed70995f6ae54349b228842cc541b68d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 46421 } } peers { permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 } } peers { permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 } }
I20260620 18:48:21.532822 2370 leader_election.cc:288] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [CANDIDATE]: Term 1 election: Requested vote from peers d3575ee3ea5c47f9906608f79a353c2e (127.15.77.65:40507), 0a21a4d7cc9f4021922f4262f4db5bb1 (127.15.77.66:44291)
I20260620 18:48:21.532991 2040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3804433eb5a40d38057f973b38af0cd" candidate_uuid: "ed70995f6ae54349b228842cc541b68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3575ee3ea5c47f9906608f79a353c2e"
I20260620 18:48:21.533012 2171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3804433eb5a40d38057f973b38af0cd" candidate_uuid: "ed70995f6ae54349b228842cc541b68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1"
I20260620 18:48:21.533080 2171 raft_consensus.cc:3037] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:21.533082 2040 raft_consensus.cc:3037] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:21.533995 2171 raft_consensus.cc:2454] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed70995f6ae54349b228842cc541b68d in term 1.
I20260620 18:48:21.533995 2040 raft_consensus.cc:2454] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed70995f6ae54349b228842cc541b68d in term 1.
I20260620 18:48:21.534166 2238 leader_election.cc:302] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a21a4d7cc9f4021922f4262f4db5bb1, ed70995f6ae54349b228842cc541b68d; no voters:
I20260620 18:48:21.534257 2370 raft_consensus.cc:2781] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:21.534413 2370 raft_consensus.cc:686] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 1 LEADER]: Becoming Leader. State: Replica: ed70995f6ae54349b228842cc541b68d, State: Running, Role: LEADER
I20260620 18:48:21.534528 2370 consensus_queue.cc:237] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed70995f6ae54349b228842cc541b68d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 46421 } } peers { permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 } } peers { permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 } }
I20260620 18:48:21.535456 1897 catalog_manager.cc:5495] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d reported cstate change: term changed from 0 to 1, leader changed from <none> to ed70995f6ae54349b228842cc541b68d (127.15.77.67). New cstate: current_term: 1 leader_uuid: "ed70995f6ae54349b228842cc541b68d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed70995f6ae54349b228842cc541b68d" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 46421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 } health_report { overall_health: UNKNOWN } } }
I20260620 18:48:21.545693 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.543269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.15.77.67:46421)
0620 18:48:21.543358 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:21.543360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:21.543370 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:48:21.543419 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:21.543419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:21.543437 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:21.543523 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:21.543687 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.544156 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.544938 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:21.545133 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:21.545208 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:21.545353 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:21.545440 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:21.545551 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:21.545581 (+ 30us) server_negotiation.cc:299] Negotiation successful
0620 18:48:21.545590 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 18:48:21.547720 2171 raft_consensus.cc:1261] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1 [term 1 FOLLOWER]: Refusing update from remote peer ed70995f6ae54349b228842cc541b68d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:21.547720 2040 raft_consensus.cc:1261] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e [term 1 FOLLOWER]: Refusing update from remote peer ed70995f6ae54349b228842cc541b68d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:21.548055 2370 consensus_queue.cc:1035] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a21a4d7cc9f4021922f4262f4db5bb1" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:21.548200 2372 consensus_queue.cc:1035] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3575ee3ea5c47f9906608f79a353c2e" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:21.551190 2377 mvcc.cc:205] Tried to move back new op lower bound from 7298995411136487424 to 7298995411086102528. Current Snapshot: MvccSnapshot[applied={T|T < 7298995411136487424 or (T in {7298995411136487424})}]
I20260620 18:48:21.551527 2376 mvcc.cc:205] Tried to move back new op lower bound from 7298995411136487424 to 7298995411086102528. Current Snapshot: MvccSnapshot[applied={T|T < 7298995411136487424}]
I20260620 18:48:21.552802 1897 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:21.552886 1897 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:21.553699 1897 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b25e7607f29040bb85de8f144846f574: Sending DeleteTablet for 3 replicas of tablet c3804433eb5a40d38057f973b38af0cd
I20260620 18:48:21.554040 2282 tablet_service.cc:1514] Processing DeleteTablet for tablet c3804433eb5a40d38057f973b38af0cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:21 UTC) from {username='slave'} at 127.0.0.1:49754
I20260620 18:48:21.554090 2020 tablet_service.cc:1514] Processing DeleteTablet for tablet c3804433eb5a40d38057f973b38af0cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:21 UTC) from {username='slave'} at 127.0.0.1:39316
I20260620 18:48:21.554308 2151 tablet_service.cc:1514] Processing DeleteTablet for tablet c3804433eb5a40d38057f973b38af0cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:21 UTC) from {username='slave'} at 127.0.0.1:48918
I20260620 18:48:21.554308 2385 tablet_replica.cc:330] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e: stopping tablet replica
I20260620 18:48:21.554515 2385 raft_consensus.cc:2229] T c3804433eb5a40d38057f973b38af0cd P d3575ee3ea5c47f9906608f79a353c2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:48:21.554526 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 1958
I20260620 18:48:21.554540 2386 tablet_replica.cc:330] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: stopping tablet replica
I20260620 18:48:21.555404 2386 raft_consensus.cc:2229] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:48:21.555529 2386 raft_consensus.cc:2258] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:21.555917 2386 ts_tablet_manager.cc:1905] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:21.557399 2386 ts_tablet_manager.cc:1918] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:21.557502 2386 log.cc:1196] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/c3804433eb5a40d38057f973b38af0cd
I20260620 18:48:21.557745 2386 ts_tablet_manager.cc:1939] T c3804433eb5a40d38057f973b38af0cd P 0a21a4d7cc9f4021922f4262f4db5bb1: Deleting consensus metadata
I20260620 18:48:21.558166 1885 catalog_manager.cc:4841] TS 0a21a4d7cc9f4021922f4262f4db5bb1 (127.15.77.66:44291): tablet c3804433eb5a40d38057f973b38af0cd (table test-table [id=27b86ce564904f4ead1b465bf9a04321]) successfully deleted
I20260620 18:48:21.560122 2384 tablet_replica.cc:330] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: stopping tablet replica
I20260620 18:48:21.560266 2384 raft_consensus.cc:2229] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:48:21.560402 2384 raft_consensus.cc:2258] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 18:48:21.562546 1885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:40507 (error 108)
I20260620 18:48:21.562620 2384 ts_tablet_manager.cc:1905] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:21.562775 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 2089
I20260620 18:48:21.563220 2360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:21.563093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:40507 (local address 127.0.0.1:39332)
0620 18:48:21.563139 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:21.563180 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:40507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260620 18:48:21.563354 1885 catalog_manager.cc:4584] TS d3575ee3ea5c47f9906608f79a353c2e (127.15.77.65:40507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c3804433eb5a40d38057f973b38af0cd: Network error: Client connection negotiation failed: client connection to 127.15.77.65:40507: connect: Connection refused (error 111)
I20260620 18:48:21.564653 2384 ts_tablet_manager.cc:1918] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:21.564742 2384 log.cc:1196] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/c3804433eb5a40d38057f973b38af0cd
I20260620 18:48:21.564997 2384 ts_tablet_manager.cc:1939] T c3804433eb5a40d38057f973b38af0cd P ed70995f6ae54349b228842cc541b68d: Deleting consensus metadata
I20260620 18:48:21.565405 1885 catalog_manager.cc:4841] TS ed70995f6ae54349b228842cc541b68d (127.15.77.67:46421): tablet c3804433eb5a40d38057f973b38af0cd (table test-table [id=27b86ce564904f4ead1b465bf9a04321]) successfully deleted
I20260620 18:48:21.570722 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 2220
I20260620 18:48:21.575698 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 1867
2026-06-20T18:48:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (759 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:21 dist-test-slave-tvl9 krb5kdc[2390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:21 dist-test-slave-tvl9 krb5kdc[2390](info): set up 2 sockets
Jun 20 18:48:21 dist-test-slave-tvl9 krb5kdc[2390](info): commencing operation
krb5kdc: starting...
W20260620 18:48:22.615298 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:22 dist-test-slave-tvl9 krb5kdc[2390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:22.735100 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:41861
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:41861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:22.817226 2403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:22.817443 2403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:22.817466 2403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:22.818526 2403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:22.818559 2403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:22.818573 2403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:22.818585 2403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:22.818596 2403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:22.820147 2403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:41861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:41861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:22.820364 2403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:22.820544 2403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:22.825574 2409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:22.825654 2403 server_base.cc:1034] running on GCE node
W20260620 18:48:22.825596 2408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:22.825596 2411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:22.826130 2403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:22.826453 2403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:22.827625 2403 hybrid_clock.cc:648] HybridClock initialized: now 1781981302827612 us; error 29 us; skew 500 ppm
Jun 20 18:48:22 dist-test-slave-tvl9 krb5kdc[2390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981302, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:22.830276 2403 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:22.830739 2403 webserver.cc:460] Webserver started at http://127.15.77.126:44969/ using document root <none> and password file <none>
I20260620 18:48:22.830997 2403 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:22.831068 2403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:22.831218 2403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:22.832361 2403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "e6b3294a74934703925e19654d36b0f6"
format_stamp: "Formatted at 2026-06-20 18:48:22 on dist-test-slave-tvl9"
server_key: "476c3e6b60d85e2d8733a4293e7853b8"
server_key_iv: "97715eab62ec490b518e209590914429"
server_key_version: "encryptionkey@0"
I20260620 18:48:22.832736 2403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "e6b3294a74934703925e19654d36b0f6"
format_stamp: "Formatted at 2026-06-20 18:48:22 on dist-test-slave-tvl9"
server_key: "476c3e6b60d85e2d8733a4293e7853b8"
server_key_iv: "97715eab62ec490b518e209590914429"
server_key_version: "encryptionkey@0"
I20260620 18:48:22.834116 2403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:48:22.834885 2418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:22.835098 2403 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:22.835206 2403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "e6b3294a74934703925e19654d36b0f6"
format_stamp: "Formatted at 2026-06-20 18:48:22 on dist-test-slave-tvl9"
server_key: "476c3e6b60d85e2d8733a4293e7853b8"
server_key_iv: "97715eab62ec490b518e209590914429"
server_key_version: "encryptionkey@0"
I20260620 18:48:22.835289 2403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:22.844933 2403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:22.847298 2403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:22.847528 2403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:22.851574 2403 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:41861
I20260620 18:48:22.851624 2470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:41861 every 8 connection(s)
I20260620 18:48:22.852005 2403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:22.852869 2471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:22.855167 2471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6: Bootstrap starting.
I20260620 18:48:22.855736 2471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:22.856060 2471 log.cc:824] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:22.856736 2471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6: No bootstrap required, opened a new log
I20260620 18:48:22.858493 2471 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b3294a74934703925e19654d36b0f6" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41861 } }
I20260620 18:48:22.858632 2471 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:22.858655 2471 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b3294a74934703925e19654d36b0f6, State: Initialized, Role: FOLLOWER
I20260620 18:48:22.858798 2471 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b3294a74934703925e19654d36b0f6" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41861 } }
I20260620 18:48:22.858855 2471 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:22.859061 2474 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:22.859151 2474 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b3294a74934703925e19654d36b0f6" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41861 } }
I20260620 18:48:22.859283 2471 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:22.859289 2474 leader_election.cc:302] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6b3294a74934703925e19654d36b0f6; no voters:
I20260620 18:48:22.859395 2471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:22.859557 2474 leader_election.cc:288] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260620 18:48:22.860071 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 2403
I20260620 18:48:22.860082 2471 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b3294a74934703925e19654d36b0f6" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41861 } }
I20260620 18:48:22.860163 2471 leader_election.cc:302] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6b3294a74934703925e19654d36b0f6; no voters:
I20260620 18:48:22.860184 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:22.860216 2471 leader_election.cc:288] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:22.860234 2474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:22.860397 15669 external_mini_cluster.cc:1428] Setting key 6d4614414af27407ad198e0314527992
I20260620 18:48:22.860491 2474 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 1 LEADER]: Becoming Leader. State: Replica: e6b3294a74934703925e19654d36b0f6, State: Running, Role: LEADER
I20260620 18:48:22.860496 2471 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:22.860632 2474 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b3294a74934703925e19654d36b0f6" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41861 } }
I20260620 18:48:22.861052 2475 raft_consensus.cc:2741] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [term 1 LEADER]: Leader pre-election decision vote started in defunct term 0: won
I20260620 18:48:22.861222 2477 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6b3294a74934703925e19654d36b0f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b3294a74934703925e19654d36b0f6" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41861 } } }
I20260620 18:48:22.861258 2476 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6b3294a74934703925e19654d36b0f6. Latest consensus state: current_term: 1 leader_uuid: "e6b3294a74934703925e19654d36b0f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b3294a74934703925e19654d36b0f6" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41861 } } }
I20260620 18:48:22.861322 2477 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:22.861339 2476 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:22.861528 2485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:22.862087 2485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:22.864030 2485 catalog_manager.cc:1347] Generated new cluster ID: 6e4f53cdd85f474b8d7d83cf62aa7df1
I20260620 18:48:22.864084 2485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:48:22 dist-test-slave-tvl9 krb5kdc[2390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:22.872736 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:22.861422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49388 (local address 127.15.77.126:41861)
0620 18:48:22.861581 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:22.861591 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:22.861626 (+ 35us) server_negotiation.cc:407] Connection header received
0620 18:48:22.861751 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:22.861758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:22.861816 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:22.862786 (+ 970us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:22.862882 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:22.864260 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:22.865323 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:22.865975 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:22.866096 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:22.868987 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:22.868991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:22.868992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:22.869008 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:22.870982 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:22.871427 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:22.871432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:22.871432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:22.871569 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:22.871783 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:22.871784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:22.871785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:22.872093 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:22.872258 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:22.872333 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 18:48:22.872487 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260620 18:48:22.879069 2485 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:22.879834 2485 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:22.885778 2485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6: Generated new TSK 0
I20260620 18:48:22.885982 2485 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.77.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
Entry for principal kudu/127.15.77.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab.
I20260620 18:48:22.918840 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41861
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:23.001076 2498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:23.001277 2498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:23.001300 2498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:23.002449 2498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:23.002489 2498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:23.002542 2498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:48:23.004081 2498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.65.keytab
--principal=kudu/127.15.77.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:23.004328 2498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:23.004602 2498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:23.009744 2506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:23.009794 2498 server_base.cc:1034] running on GCE node
W20260620 18:48:23.009747 2504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:23.009747 2503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:23.010218 2498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:23.010480 2498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:23.011622 2498 hybrid_clock.cc:648] HybridClock initialized: now 1781981303011608 us; error 34 us; skew 500 ppm
Jun 20 18:48:23 dist-test-slave-tvl9 krb5kdc[2390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:23.014225 2498 init.cc:377] Logged in from keytab as kudu/127.15.77.65@KRBTEST.COM (short username kudu)
I20260620 18:48:23.014662 2498 webserver.cc:460] Webserver started at http://127.15.77.65:44607/ using document root <none> and password file <none>
I20260620 18:48:23.014891 2498 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:23.014961 2498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:23.015120 2498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:23.016237 2498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "29c6e595ee454b95830b5044cf250565"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "6a12b1cb8cccba978b7d342762467289"
server_key_iv: "f082519074c62345ece3d268e6d25938"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.016608 2498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "29c6e595ee454b95830b5044cf250565"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "6a12b1cb8cccba978b7d342762467289"
server_key_iv: "f082519074c62345ece3d268e6d25938"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.018016 2498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:48:23.018767 2513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.018956 2498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:23.019061 2498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "29c6e595ee454b95830b5044cf250565"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "6a12b1cb8cccba978b7d342762467289"
server_key_iv: "f082519074c62345ece3d268e6d25938"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.019148 2498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:23.043810 2498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:23.046293 2498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:23.046464 2498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:23.046721 2498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:23.047055 2498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:23.047109 2498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.047151 2498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:23.047180 2498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.052819 2498 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:34257
I20260620 18:48:23.052862 2626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:34257 every 8 connection(s)
I20260620 18:48:23.053175 2498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:48:23.053763 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 2498
I20260620 18:48:23.053896 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:48:23.054131 15669 external_mini_cluster.cc:1428] Setting key 40389be1a6e690bda1571e0d486c58a3
Jun 20 18:48:23 dist-test-slave-tvl9 krb5kdc[2390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.65@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.66@KRBTEST.COM; defaulting to no policy
I20260620 18:48:23.063567 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.053908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:45899 (local address 127.15.77.126:41861)
0620 18:48:23.053997 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.054000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.054225 (+ 225us) server_negotiation.cc:407] Connection header received
0620 18:48:23.054871 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.054872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.054886 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.054949 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:23.055987 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.056528 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.057533 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.057743 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.057818 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.060297 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:23.060299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:23.060299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:23.060309 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:23.062018 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:23.062492 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:23.062493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:23.062494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:23.062556 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:23.062819 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:23.062820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:23.062820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:23.062946 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:23.063027 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.063364 (+ 337us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.063465 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 18:48:23.063736 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.053890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41861 (local address 127.15.77.65:45899)
0620 18:48:23.054106 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.054148 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:23.054763 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:23.055026 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:23.055031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:23.055687 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:23.055881 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.055885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.056631 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.056632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.057428 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.057430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.057826 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.057827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.057956 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.058498 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:23.058507 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:23.060172 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:23.062128 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:23.062136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:23.062139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:23.062395 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:23.062641 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:23.062641 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:23.062642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:23.062749 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:23.063032 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:23.063040 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:23.063178 (+ 138us) client_negotiation.cc:769] Sending connection context
0620 18:48:23.063281 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 18:48:23.063496 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":123,"threads_started":1}
I20260620 18:48:23.064256 2627 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41861
I20260620 18:48:23.064373 2627 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:23.064561 2627 heartbeater.cc:511] Master 127.15.77.126:41861 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.66@KRBTEST.COM" created.
I20260620 18:48:23.065073 2435 ts_manager.cc:194] Registered new tserver with Master: 29c6e595ee454b95830b5044cf250565 (127.15.77.65:34257)
I20260620 18:48:23.065953 2435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.65@KRBTEST.COM'} at 127.15.77.65:45899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
Entry for principal kudu/127.15.77.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab.
I20260620 18:48:23.082728 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41861
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:23.165536 2632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:23.165745 2632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:23.165769 2632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:23.166859 2632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:23.166898 2632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:23.166923 2632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:48:23.168440 2632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.66.keytab
--principal=kudu/127.15.77.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:23.168642 2632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:23.168823 2632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:48:23.174376 2632 server_base.cc:1034] running on GCE node
W20260620 18:48:23.174289 2640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:23.174304 2637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:23.174324 2638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:23.174757 2632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:23.175045 2632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:23.176201 2632 hybrid_clock.cc:648] HybridClock initialized: now 1781981303176178 us; error 39 us; skew 500 ppm
Jun 20 18:48:23 dist-test-slave-tvl9 krb5kdc[2390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:23.178736 2632 init.cc:377] Logged in from keytab as kudu/127.15.77.66@KRBTEST.COM (short username kudu)
I20260620 18:48:23.179179 2632 webserver.cc:460] Webserver started at http://127.15.77.66:34185/ using document root <none> and password file <none>
I20260620 18:48:23.179491 2632 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:23.179567 2632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:23.179714 2632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:23.180788 2632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "298ead6ccbd443e3b65b32e9f1936f82"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "8c099b2ed6edced8cc1d97e871df9190"
server_key_iv: "45d9e75015157a59a9d0811b678c21cf"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.181149 2632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "298ead6ccbd443e3b65b32e9f1936f82"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "8c099b2ed6edced8cc1d97e871df9190"
server_key_iv: "45d9e75015157a59a9d0811b678c21cf"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.182500 2632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:23.183269 2647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.183528 2632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:23.183631 2632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "298ead6ccbd443e3b65b32e9f1936f82"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "8c099b2ed6edced8cc1d97e871df9190"
server_key_iv: "45d9e75015157a59a9d0811b678c21cf"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.183718 2632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:23.203991 2632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:23.205912 2632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:23.206095 2632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:23.206332 2632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:23.206679 2632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:23.206738 2632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.206780 2632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:23.206809 2632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.212416 2632 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:42379
I20260620 18:48:23.212491 2760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:42379 every 8 connection(s)
I20260620 18:48:23.212828 2632 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:48:23.217935 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 2632
I20260620 18:48:23.218070 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:48:23.218274 15669 external_mini_cluster.cc:1428] Setting key a623b104fcc7e4f2e637bdc25bf5bbba
Jun 20 18:48:23 dist-test-slave-tvl9 krb5kdc[2390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.66@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:23.223796 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.213596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:43551 (local address 127.15.77.126:41861)
0620 18:48:23.213665 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.213667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.213880 (+ 213us) server_negotiation.cc:407] Connection header received
0620 18:48:23.214381 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.214383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.214395 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.214459 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:23.215520 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.216051 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.217061 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.217266 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.217352 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.219866 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:23.219867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:23.219868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:23.219876 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:23.221616 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:23.222343 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:23.222345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:23.222345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:23.222431 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:23.222810 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:23.222811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:23.222812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:23.223004 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:23.223096 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.223536 (+ 440us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.223660 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 18:48:23.223898 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.213558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41861 (local address 127.15.77.66:43551)
0620 18:48:23.213747 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.213789 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:23.214261 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:23.214528 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:23.214532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:23.215162 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:23.215422 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.215428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.216161 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.216162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.216968 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.216970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.217364 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.217365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.217498 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.218132 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:23.218144 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:23.219769 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:23.221738 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:23.221751 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:23.221753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:23.222240 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:23.222510 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:23.222510 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:23.222511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:23.222664 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:23.223092 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:23.223100 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:23.223306 (+ 206us) client_negotiation.cc:769] Sending connection context
0620 18:48:23.223478 (+ 172us) client_negotiation.cc:240] Negotiation successful
0620 18:48:23.223660 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260620 18:48:23.226238 2761 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41861
I20260620 18:48:23.226423 2761 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:23.226785 2761 heartbeater.cc:511] Master 127.15.77.126:41861 requested a full tablet report, sending...
I20260620 18:48:23.228370 2435 ts_manager.cc:194] Registered new tserver with Master: 298ead6ccbd443e3b65b32e9f1936f82 (127.15.77.66:42379)
WARNING: no policy specified for kudu/127.15.77.67@KRBTEST.COM; defaulting to no policy
I20260620 18:48:23.229020 2435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.66@KRBTEST.COM'} at 127.15.77.66:43551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
Entry for principal kudu/127.15.77.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab.
I20260620 18:48:23.246744 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41861
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:23.328305 2766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:23.328518 2766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:23.328541 2766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:23.329613 2766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:23.329646 2766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:23.329671 2766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:48:23.331039 2766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.67.keytab
--principal=kudu/127.15.77.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:23.331223 2766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:23.331472 2766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:23.336295 2772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:23.336265 2771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:23.336354 2766 server_base.cc:1034] running on GCE node
W20260620 18:48:23.336264 2774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:23.336685 2766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:23.336879 2766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:23.338042 2766 hybrid_clock.cc:648] HybridClock initialized: now 1781981303338014 us; error 42 us; skew 500 ppm
Jun 20 18:48:23 dist-test-slave-tvl9 krb5kdc[2390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:23.340600 2766 init.cc:377] Logged in from keytab as kudu/127.15.77.67@KRBTEST.COM (short username kudu)
I20260620 18:48:23.341024 2766 webserver.cc:460] Webserver started at http://127.15.77.67:42801/ using document root <none> and password file <none>
I20260620 18:48:23.341255 2766 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:23.341321 2766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:23.341473 2766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:23.342537 2766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "688f53a32fd140fc9b0b9582085ccf45"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "033db7add8ca935b346eeed78065456a"
server_key_iv: "f83313d737195140b7a08dd13dc7bdbc"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.342883 2766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "688f53a32fd140fc9b0b9582085ccf45"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "033db7add8ca935b346eeed78065456a"
server_key_iv: "f83313d737195140b7a08dd13dc7bdbc"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.344372 2766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:48:23.345077 2781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.345244 2766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.345345 2766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "688f53a32fd140fc9b0b9582085ccf45"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "033db7add8ca935b346eeed78065456a"
server_key_iv: "f83313d737195140b7a08dd13dc7bdbc"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.345420 2766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:23.357969 2766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:23.359961 2766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:23.360177 2766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:23.360416 2766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:23.360729 2766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:23.360802 2766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.360841 2766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:23.360883 2766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.367317 2766 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:45389
I20260620 18:48:23.367376 2894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:45389 every 8 connection(s)
I20260620 18:48:23.367708 2766 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:48:23.371531 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 2766
I20260620 18:48:23.371673 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:48:23.371845 15669 external_mini_cluster.cc:1428] Setting key 29179d87f2e0b9711e44c4fdaa4f6f40
Jun 20 18:48:23 dist-test-slave-tvl9 krb5kdc[2390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.67@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:23.376924 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.368420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:42749 (local address 127.15.77.126:41861)
0620 18:48:23.368509 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.368514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.368685 (+ 171us) server_negotiation.cc:407] Connection header received
0620 18:48:23.369148 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.369149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.369159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.369227 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:23.370198 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.370695 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.371844 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.372107 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.372196 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.374259 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:23.374261 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:23.374261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:23.374267 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:23.375797 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:23.376149 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:23.376149 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:23.376150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:23.376200 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:23.376405 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:23.376405 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:23.376405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:23.376527 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:23.376578 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.376801 (+ 223us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.376859 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 18:48:23.377171 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.368384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41861 (local address 127.15.77.67:42749)
0620 18:48:23.368544 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.368598 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:23.369067 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:23.369281 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:23.369284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:23.369874 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:23.370068 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.370073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.370784 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.370785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.371689 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.371692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.372208 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.372209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.372337 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.372799 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:23.372810 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:23.374177 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:23.375865 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:23.375871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:23.375873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:23.376078 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:23.376256 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:23.376257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:23.376257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:23.376352 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:23.376582 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:23.376586 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:23.376696 (+ 110us) client_negotiation.cc:769] Sending connection context
0620 18:48:23.376775 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 18:48:23.376892 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260620 18:48:23.377645 2895 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41861
I20260620 18:48:23.377746 2895 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:23.377890 2895 heartbeater.cc:511] Master 127.15.77.126:41861 requested a full tablet report, sending...
I20260620 18:48:23.378262 2435 ts_manager.cc:194] Registered new tserver with Master: 688f53a32fd140fc9b0b9582085ccf45 (127.15.77.67:45389)
I20260620 18:48:23.378603 2435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.77.67@KRBTEST.COM'} at 127.15.77.67:42749
I20260620 18:48:23.383800 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:48:23.391050 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.385112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49402 (local address 127.15.77.126:41861)
0620 18:48:23.385186 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.385189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.385263 (+ 74us) server_negotiation.cc:407] Connection header received
0620 18:48:23.385304 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.385304 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.385313 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.385362 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:23.385908 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.386497 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.387216 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.387519 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.387609 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.388804 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:23.388806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:23.388806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:23.388812 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:23.390088 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:23.390409 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:23.390410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:23.390410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:23.390462 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:23.390622 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:23.390622 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:23.390623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:23.390762 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:23.390838 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.390928 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.390987 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 18:48:23.392817 2435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:48:23.394385 2435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:48:23.400782 2907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.396686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:42379 (local address 127.0.0.1:39612)
0620 18:48:23.397029 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.397038 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:23.397098 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:23.397316 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:23.397320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:23.397322 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:23.397398 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.397399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.398401 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.398402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.399530 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.399533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.400473 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.400474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.400557 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.400571 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 18:48:23.400643 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 18:48:23.400702 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260620 18:48:23.400789 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.396856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39612 (local address 127.15.77.66:42379)
0620 18:48:23.397001 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.397006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.397114 (+ 108us) server_negotiation.cc:407] Connection header received
0620 18:48:23.397167 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.397170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.397188 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.397298 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:23.397503 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.398296 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.399639 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.400369 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.400470 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.400537 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.400664 (+ 127us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.400706 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260620 18:48:23.401304 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.396686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34257 (local address 127.0.0.1:38568)
0620 18:48:23.396854 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.396919 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:23.396982 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:23.397456 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:23.397458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:23.397460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:23.397516 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.397517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.398924 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.398925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.400011 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.400013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.401059 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.401060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.401114 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.401118 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:48:23.401147 (+ 29us) client_negotiation.cc:240] Negotiation successful
0620 18:48:23.401178 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":52,"threads_started":1}
I20260620 18:48:23.401440 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.397102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55736 (local address 127.15.77.67:45389)
0620 18:48:23.397253 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.397257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.397266 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:23.397423 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.397426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.397447 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.397524 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:23.397718 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.398816 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.399974 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.400726 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.400800 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.401266 (+ 466us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.401292 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.401336 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260620 18:48:23.401588 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.396893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:45389 (local address 127.0.0.1:55736)
0620 18:48:23.397102 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.397113 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:23.397152 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:23.397614 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:23.397615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:23.397617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:23.397650 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.397652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.398963 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.398964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.399885 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.399888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.401009 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.401011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.401085 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.401089 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:48:23.401131 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 18:48:23.401175 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
I20260620 18:48:23.401619 2695 tablet_service.cc:1467] Processing CreateTablet for tablet 305529cfbfdb46298db44c632f4df771 (DEFAULT_TABLE table=test-table [id=ed44ac17c3a94c6cb7b8782d6c0a9a38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:23.401762 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.396768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38568 (local address 127.15.77.65:34257)
0620 18:48:23.397293 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.397298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.397308 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:48:23.397346 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.397348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.397361 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.397601 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:23.397685 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.398805 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.400084 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.400780 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.401223 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.401302 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.401326 (+ 24us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.401673 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":99,"threads_started":1}
I20260620 18:48:23.402036 2695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 305529cfbfdb46298db44c632f4df771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:23.402261 2829 tablet_service.cc:1467] Processing CreateTablet for tablet 305529cfbfdb46298db44c632f4df771 (DEFAULT_TABLE table=test-table [id=ed44ac17c3a94c6cb7b8782d6c0a9a38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:23.402293 2561 tablet_service.cc:1467] Processing CreateTablet for tablet 305529cfbfdb46298db44c632f4df771 (DEFAULT_TABLE table=test-table [id=ed44ac17c3a94c6cb7b8782d6c0a9a38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:23.402601 2561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 305529cfbfdb46298db44c632f4df771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:23.402601 2829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 305529cfbfdb46298db44c632f4df771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:23.404981 2912 tablet_bootstrap.cc:492] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: Bootstrap starting.
I20260620 18:48:23.405525 2912 tablet_bootstrap.cc:654] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:23.405804 2914 tablet_bootstrap.cc:492] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: Bootstrap starting.
I20260620 18:48:23.406136 2912 log.cc:824] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:23.406541 2914 tablet_bootstrap.cc:654] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:23.406860 2912 tablet_bootstrap.cc:492] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: No bootstrap required, opened a new log
I20260620 18:48:23.406924 2912 ts_tablet_manager.cc:1397] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:48:23.407094 2914 log.cc:824] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:23.407438 2913 tablet_bootstrap.cc:492] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565: Bootstrap starting.
I20260620 18:48:23.407811 2914 tablet_bootstrap.cc:492] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: No bootstrap required, opened a new log
I20260620 18:48:23.407892 2914 ts_tablet_manager.cc:1397] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 18:48:23.408043 2913 tablet_bootstrap.cc:654] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:23.408507 2913 log.cc:824] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:23.409014 2912 raft_consensus.cc:348] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298ead6ccbd443e3b65b32e9f1936f82" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42379 } } peers { permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 } } peers { permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 } }
I20260620 18:48:23.409171 2913 tablet_bootstrap.cc:492] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565: No bootstrap required, opened a new log
I20260620 18:48:23.409202 2912 raft_consensus.cc:374] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:23.409242 2913 ts_tablet_manager.cc:1397] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 18:48:23.409241 2912 raft_consensus.cc:729] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 298ead6ccbd443e3b65b32e9f1936f82, State: Initialized, Role: FOLLOWER
I20260620 18:48:23.409391 2912 consensus_queue.cc:260] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298ead6ccbd443e3b65b32e9f1936f82" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42379 } } peers { permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 } } peers { permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 } }
I20260620 18:48:23.409669 2912 ts_tablet_manager.cc:1428] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 18:48:23.409803 2761 heartbeater.cc:503] Master 127.15.77.126:41861 was elected leader, sending a full tablet report...
I20260620 18:48:23.409795 2914 raft_consensus.cc:348] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298ead6ccbd443e3b65b32e9f1936f82" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42379 } } peers { permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 } } peers { permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 } }
I20260620 18:48:23.410005 2914 raft_consensus.cc:374] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:23.410049 2914 raft_consensus.cc:729] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688f53a32fd140fc9b0b9582085ccf45, State: Initialized, Role: FOLLOWER
I20260620 18:48:23.410172 2914 consensus_queue.cc:260] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298ead6ccbd443e3b65b32e9f1936f82" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42379 } } peers { permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 } } peers { permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 } }
I20260620 18:48:23.410461 2914 ts_tablet_manager.cc:1428] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 18:48:23.410531 2895 heartbeater.cc:503] Master 127.15.77.126:41861 was elected leader, sending a full tablet report...
I20260620 18:48:23.411222 2913 raft_consensus.cc:348] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298ead6ccbd443e3b65b32e9f1936f82" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42379 } } peers { permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 } } peers { permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 } }
I20260620 18:48:23.411478 2913 raft_consensus.cc:374] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:23.411528 2913 raft_consensus.cc:729] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c6e595ee454b95830b5044cf250565, State: Initialized, Role: FOLLOWER
I20260620 18:48:23.411650 2913 consensus_queue.cc:260] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298ead6ccbd443e3b65b32e9f1936f82" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42379 } } peers { permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 } } peers { permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 } }
I20260620 18:48:23.411975 2627 heartbeater.cc:503] Master 127.15.77.126:41861 was elected leader, sending a full tablet report...
I20260620 18:48:23.411959 2913 ts_tablet_manager.cc:1428] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260620 18:48:23.463488 2762 tablet.cc:2369] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:48:23.479285 2918 raft_consensus.cc:482] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:23.479440 2918 raft_consensus.cc:504] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298ead6ccbd443e3b65b32e9f1936f82" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42379 } } peers { permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 } } peers { permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 } }
I20260620 18:48:23.479794 2918 leader_election.cc:288] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29c6e595ee454b95830b5044cf250565 (127.15.77.65:34257), 688f53a32fd140fc9b0b9582085ccf45 (127.15.77.67:45389)
I20260620 18:48:23.483810 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.480133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:40285 (local address 127.15.77.65:34257)
0620 18:48:23.480181 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.480184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.480193 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:23.480229 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.480230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.480244 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.480304 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:23.480508 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.481312 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.482806 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.483421 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.483500 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.483509 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.483699 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.483737 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:48:23.483810 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.480111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:59931 (local address 127.15.77.67:45389)
0620 18:48:23.480152 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.480155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.480369 (+ 214us) server_negotiation.cc:407] Connection header received
0620 18:48:23.480423 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.480424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.480433 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.480504 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:23.480629 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.481320 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.482806 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.483426 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.483500 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.483509 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.483699 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.483737 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 18:48:23.483829 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.480023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34257 (local address 127.15.77.66:40285)
0620 18:48:23.480090 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.480104 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:23.480165 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:23.480332 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:23.480334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:23.480335 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:23.480418 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.480420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.481403 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.481404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.482697 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.482700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.483526 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.483528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.483615 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.483617 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:48:23.483681 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 18:48:23.483730 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":61}
I20260620 18:48:23.483834 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.480023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:45389 (local address 127.15.77.66:59931)
0620 18:48:23.480305 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.480316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:23.480372 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:23.480504 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:23.480504 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:23.480506 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:23.480563 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.480564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.481403 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.481404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.482697 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.482700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.483527 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.483528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.483612 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.483617 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:48:23.483680 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 18:48:23.483730 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":92,"thread_start_us":123,"threads_started":1}
I20260620 18:48:23.484244 2581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305529cfbfdb46298db44c632f4df771" candidate_uuid: "298ead6ccbd443e3b65b32e9f1936f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c6e595ee454b95830b5044cf250565" is_pre_election: true
I20260620 18:48:23.484264 2849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305529cfbfdb46298db44c632f4df771" candidate_uuid: "298ead6ccbd443e3b65b32e9f1936f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688f53a32fd140fc9b0b9582085ccf45" is_pre_election: true
I20260620 18:48:23.484412 2581 raft_consensus.cc:2454] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 298ead6ccbd443e3b65b32e9f1936f82 in term 0.
I20260620 18:48:23.484422 2849 raft_consensus.cc:2454] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 298ead6ccbd443e3b65b32e9f1936f82 in term 0.
I20260620 18:48:23.484663 2650 leader_election.cc:302] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 298ead6ccbd443e3b65b32e9f1936f82, 688f53a32fd140fc9b0b9582085ccf45; no voters:
I20260620 18:48:23.484825 2918 raft_consensus.cc:2781] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:48:23.484903 2918 raft_consensus.cc:482] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:48:23.484922 2918 raft_consensus.cc:3037] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:23.485680 2918 raft_consensus.cc:504] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298ead6ccbd443e3b65b32e9f1936f82" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42379 } } peers { permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 } } peers { permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 } }
I20260620 18:48:23.485833 2918 leader_election.cc:288] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [CANDIDATE]: Term 1 election: Requested vote from peers 29c6e595ee454b95830b5044cf250565 (127.15.77.65:34257), 688f53a32fd140fc9b0b9582085ccf45 (127.15.77.67:45389)
I20260620 18:48:23.485996 2581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305529cfbfdb46298db44c632f4df771" candidate_uuid: "298ead6ccbd443e3b65b32e9f1936f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c6e595ee454b95830b5044cf250565"
I20260620 18:48:23.486020 2849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305529cfbfdb46298db44c632f4df771" candidate_uuid: "298ead6ccbd443e3b65b32e9f1936f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688f53a32fd140fc9b0b9582085ccf45"
I20260620 18:48:23.486084 2581 raft_consensus.cc:3037] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:23.486084 2849 raft_consensus.cc:3037] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:23.486984 2581 raft_consensus.cc:2454] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 298ead6ccbd443e3b65b32e9f1936f82 in term 1.
I20260620 18:48:23.486999 2849 raft_consensus.cc:2454] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 298ead6ccbd443e3b65b32e9f1936f82 in term 1.
I20260620 18:48:23.487138 2650 leader_election.cc:302] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 298ead6ccbd443e3b65b32e9f1936f82, 688f53a32fd140fc9b0b9582085ccf45; no voters:
I20260620 18:48:23.487251 2918 raft_consensus.cc:2781] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:23.487484 2918 raft_consensus.cc:686] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 1 LEADER]: Becoming Leader. State: Replica: 298ead6ccbd443e3b65b32e9f1936f82, State: Running, Role: LEADER
I20260620 18:48:23.487600 2918 consensus_queue.cc:237] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298ead6ccbd443e3b65b32e9f1936f82" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42379 } } peers { permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 } } peers { permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 } }
I20260620 18:48:23.488413 2435 catalog_manager.cc:5495] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 298ead6ccbd443e3b65b32e9f1936f82 (127.15.77.66). New cstate: current_term: 1 leader_uuid: "298ead6ccbd443e3b65b32e9f1936f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "298ead6ccbd443e3b65b32e9f1936f82" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 } health_report { overall_health: UNKNOWN } } }
I20260620 18:48:23.516105 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.513469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39622 (local address 127.15.77.66:42379)
0620 18:48:23.513540 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.513542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.513550 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:48:23.513591 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.513592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.513612 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.513676 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:23.513845 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.514360 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.515179 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.515454 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.515521 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.515700 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:23.515809 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:23.515881 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.515921 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.515991 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 18:48:23.518181 2581 raft_consensus.cc:1261] T 305529cfbfdb46298db44c632f4df771 P 29c6e595ee454b95830b5044cf250565 [term 1 FOLLOWER]: Refusing update from remote peer 298ead6ccbd443e3b65b32e9f1936f82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:23.518210 2849 raft_consensus.cc:1261] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45 [term 1 FOLLOWER]: Refusing update from remote peer 298ead6ccbd443e3b65b32e9f1936f82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:23.518597 2918 consensus_queue.cc:1035] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688f53a32fd140fc9b0b9582085ccf45" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 45389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:23.518745 2922 consensus_queue.cc:1035] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c6e595ee454b95830b5044cf250565" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 34257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:23.521178 2925 mvcc.cc:205] Tried to move back new op lower bound from 7298995419207323648 to 7298995419085725696. Current Snapshot: MvccSnapshot[applied={T|T < 7298995419207323648}]
I20260620 18:48:23.521797 2927 mvcc.cc:205] Tried to move back new op lower bound from 7298995419207323648 to 7298995419085725696. Current Snapshot: MvccSnapshot[applied={T|T < 7298995419207323648}]
I20260620 18:48:23.522248 2926 mvcc.cc:205] Tried to move back new op lower bound from 7298995419207323648 to 7298995419085725696. Current Snapshot: MvccSnapshot[applied={T|T < 7298995419207323648}]
I20260620 18:48:23.523658 2434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:23.523741 2434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:23.524605 2434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6b3294a74934703925e19654d36b0f6: Sending DeleteTablet for 3 replicas of tablet 305529cfbfdb46298db44c632f4df771
I20260620 18:48:23.524938 2561 tablet_service.cc:1514] Processing DeleteTablet for tablet 305529cfbfdb46298db44c632f4df771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:23 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:38568
I20260620 18:48:23.524958 2829 tablet_service.cc:1514] Processing DeleteTablet for tablet 305529cfbfdb46298db44c632f4df771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:23 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:55736
I20260620 18:48:23.525219 2936 tablet_replica.cc:330] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: stopping tablet replica
I20260620 18:48:23.525305 2695 tablet_service.cc:1514] Processing DeleteTablet for tablet 305529cfbfdb46298db44c632f4df771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:23 UTC) from {username='kudu', principal='kudu/127.15.77.126@KRBTEST.COM'} at 127.0.0.1:39612
I20260620 18:48:23.525334 2936 raft_consensus.cc:2229] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:48:23.525444 2936 raft_consensus.cc:2258] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:23.525496 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 2498
I20260620 18:48:23.526592 2936 ts_tablet_manager.cc:1905] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:23.525563 2937 tablet_replica.cc:330] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: stopping tablet replica
I20260620 18:48:23.528386 2937 raft_consensus.cc:2229] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 1 LEADER]: Raft consensus shutting down.
I20260620 18:48:23.528412 2936 ts_tablet_manager.cc:1918] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:23.528493 2936 log.cc:1196] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/305529cfbfdb46298db44c632f4df771
I20260620 18:48:23.528505 2937 raft_consensus.cc:2258] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:23.528810 2936 ts_tablet_manager.cc:1939] T 305529cfbfdb46298db44c632f4df771 P 688f53a32fd140fc9b0b9582085ccf45: Deleting consensus metadata
I20260620 18:48:23.528921 2937 ts_tablet_manager.cc:1905] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:23.529250 2421 catalog_manager.cc:4841] TS 688f53a32fd140fc9b0b9582085ccf45 (127.15.77.67:45389): tablet 305529cfbfdb46298db44c632f4df771 (table test-table [id=ed44ac17c3a94c6cb7b8782d6c0a9a38]) successfully deleted
I20260620 18:48:23.530964 2937 ts_tablet_manager.cc:1918] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:23.531041 2937 log.cc:1196] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/305529cfbfdb46298db44c632f4df771
I20260620 18:48:23.531350 2937 ts_tablet_manager.cc:1939] T 305529cfbfdb46298db44c632f4df771 P 298ead6ccbd443e3b65b32e9f1936f82: Deleting consensus metadata
I20260620 18:48:23.531749 2421 catalog_manager.cc:4841] TS 298ead6ccbd443e3b65b32e9f1936f82 (127.15.77.66:42379): tablet 305529cfbfdb46298db44c632f4df771 (table test-table [id=ed44ac17c3a94c6cb7b8782d6c0a9a38]) successfully deleted
W20260620 18:48:23.532089 2419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:34257 (error 108)
I20260620 18:48:23.532338 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 2632
I20260620 18:48:23.532948 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.532531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:34257 (local address 127.0.0.1:38570)
0620 18:48:23.532868 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.532907 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:34257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314}
W20260620 18:48:23.533354 2419 catalog_manager.cc:4584] TS 29c6e595ee454b95830b5044cf250565 (127.15.77.65:34257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 305529cfbfdb46298db44c632f4df771: Network error: Client connection negotiation failed: client connection to 127.15.77.65:34257: connect: Connection refused (error 111)
I20260620 18:48:23.538097 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 2766
I20260620 18:48:23.543164 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 2403
2026-06-20T18:48:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1968 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-20T18:48:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:23Z Disabled control of system clock
I20260620 18:48:23.570910 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:41711
--webserver_interface=127.15.77.126
--webserver_port=0
--builtin_ntp_servers=127.15.77.84:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:41711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 18:48:23.650879 2942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:23.651115 2942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:23.651136 2942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:23.652328 2942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:23.652370 2942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:23.652383 2942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:23.652395 2942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:23.652407 2942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:23.653716 2942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:41711
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:41711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:23.653915 2942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:23.654166 2942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:23.659093 2948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:23.659150 2950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:23.659093 2947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:23.659446 2942 server_base.cc:1034] running on GCE node
I20260620 18:48:23.659663 2942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:23.659896 2942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:23.661033 2942 hybrid_clock.cc:648] HybridClock initialized: now 1781981303661022 us; error 27 us; skew 500 ppm
I20260620 18:48:23.662225 2942 webserver.cc:460] Webserver started at http://127.15.77.126:36999/ using document root <none> and password file <none>
I20260620 18:48:23.662415 2942 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:23.662456 2942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:23.662556 2942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:23.663662 2942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "eb43b8a2722e467ca19d0bdab2eaa33e"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "a9f441117a4e0836821a95a482df774d"
server_key_iv: "6c1967cdd534347407f992e4991b2b29"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.663961 2942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "eb43b8a2722e467ca19d0bdab2eaa33e"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "a9f441117a4e0836821a95a482df774d"
server_key_iv: "6c1967cdd534347407f992e4991b2b29"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.665170 2942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260620 18:48:23.665788 2956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.665970 2942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.666038 2942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "eb43b8a2722e467ca19d0bdab2eaa33e"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "a9f441117a4e0836821a95a482df774d"
server_key_iv: "6c1967cdd534347407f992e4991b2b29"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.666087 2942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:23.672852 2942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:23.673094 2942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:23.673199 2942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:23.677102 2942 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:41711
I20260620 18:48:23.677131 3008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:41711 every 8 connection(s)
I20260620 18:48:23.677465 2942 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:23.678421 3009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:23.680706 3009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e: Bootstrap starting.
I20260620 18:48:23.681238 3009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:23.681473 3009 log.cc:824] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:23.682116 3009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e: No bootstrap required, opened a new log
I20260620 18:48:23.683761 3009 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb43b8a2722e467ca19d0bdab2eaa33e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41711 } }
I20260620 18:48:23.683894 3009 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:23.683917 3009 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb43b8a2722e467ca19d0bdab2eaa33e, State: Initialized, Role: FOLLOWER
I20260620 18:48:23.684058 3009 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb43b8a2722e467ca19d0bdab2eaa33e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41711 } }
I20260620 18:48:23.684144 3009 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:23.684166 3009 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:23.684194 3009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:23.684665 3009 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb43b8a2722e467ca19d0bdab2eaa33e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41711 } }
I20260620 18:48:23.684744 3009 leader_election.cc:302] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb43b8a2722e467ca19d0bdab2eaa33e; no voters:
I20260620 18:48:23.684840 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 2942
I20260620 18:48:23.684930 3009 leader_election.cc:288] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:23.684965 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:23.684965 3012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:23.685086 3012 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [term 1 LEADER]: Becoming Leader. State: Replica: eb43b8a2722e467ca19d0bdab2eaa33e, State: Running, Role: LEADER
I20260620 18:48:23.685177 15669 external_mini_cluster.cc:1428] Setting key 83de6b3b5064221ca830bf8ea8f55d67
I20260620 18:48:23.685199 3009 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:23.685176 3012 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb43b8a2722e467ca19d0bdab2eaa33e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41711 } }
I20260620 18:48:23.685614 3012 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb43b8a2722e467ca19d0bdab2eaa33e. Latest consensus state: current_term: 1 leader_uuid: "eb43b8a2722e467ca19d0bdab2eaa33e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb43b8a2722e467ca19d0bdab2eaa33e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41711 } } }
I20260620 18:48:23.685699 3012 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:23.685913 3013 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb43b8a2722e467ca19d0bdab2eaa33e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb43b8a2722e467ca19d0bdab2eaa33e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 41711 } } }
I20260620 18:48:23.686019 3020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:23.686033 3013 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:23.686554 3020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:23.688248 3020 catalog_manager.cc:1347] Generated new cluster ID: 027f9aa8cab84fef9d101f93c34020d8
I20260620 18:48:23.688302 3020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:23.692236 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.685628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.15.77.126:41711)
0620 18:48:23.686487 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.686496 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.686730 (+ 234us) server_negotiation.cc:407] Connection header received
0620 18:48:23.686847 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.686855 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.686926 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.687073 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:23.687296 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.688745 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.689860 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.690456 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.691153 (+ 697us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.691533 (+ 380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:23.691537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:23.691537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:23.691733 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:23.691771 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:23.691785 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:23.691855 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.691942 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.692023 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":61,"threads_started":1}
I20260620 18:48:23.734722 3020 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:23.735280 3020 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:23.742367 3020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e: Generated new TSK 0
I20260620 18:48:23.742585 3020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:23.746009 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.65:0
--local_ip_for_outbound_sockets=127.15.77.65
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41711
--builtin_ntp_servers=127.15.77.84:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 18:48:23.828102 3033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:23.828333 3033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:23.828356 3033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:23.829576 3033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:23.829615 3033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:23.829643 3033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.65
I20260620 18:48:23.831111 3033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.77.65
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.65
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:23.831424 3033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:23.831761 3033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:23.836679 3038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:23.836699 3039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:23.836728 3041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:23.837016 3033 server_base.cc:1034] running on GCE node
I20260620 18:48:23.837188 3033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:23.837387 3033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:23.838533 3033 hybrid_clock.cc:648] HybridClock initialized: now 1781981303838522 us; error 25 us; skew 500 ppm
I20260620 18:48:23.839808 3033 webserver.cc:460] Webserver started at http://127.15.77.65:46719/ using document root <none> and password file <none>
I20260620 18:48:23.840015 3033 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:23.840057 3033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:23.840147 3033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:23.841164 3033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/data/instance:
uuid: "90babd83a3e940dabdd1c54f0ba8505d"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "1ece179bfca9b5d1b6a7eec343fc3d75"
server_key_iv: "98fdb670ac2667fae261fb1149228b36"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.841459 3033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance:
uuid: "90babd83a3e940dabdd1c54f0ba8505d"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "1ece179bfca9b5d1b6a7eec343fc3d75"
server_key_iv: "98fdb670ac2667fae261fb1149228b36"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.842692 3033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:23.843446 3047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.843634 3033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:23.843770 3033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/wal
uuid: "90babd83a3e940dabdd1c54f0ba8505d"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "1ece179bfca9b5d1b6a7eec343fc3d75"
server_key_iv: "98fdb670ac2667fae261fb1149228b36"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.843850 3033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:23.854614 3033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:23.854916 3033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:23.855064 3033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:23.855301 3033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:23.855695 3033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:23.855736 3033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.855757 3033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:23.855770 3033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.861325 3033 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.65:35453
I20260620 18:48:23.861413 3160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.65:35453 every 8 connection(s)
I20260620 18:48:23.861739 3033 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/data/info.pb
I20260620 18:48:23.867076 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.862446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:48371 (local address 127.15.77.126:41711)
0620 18:48:23.862516 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.862518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.862775 (+ 257us) server_negotiation.cc:407] Connection header received
0620 18:48:23.863404 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.863406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.863421 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.863478 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:23.864159 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.864696 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.865694 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.865906 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.865994 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.866763 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:23.866765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:23.866765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:23.866771 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:23.866785 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:23.866790 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:23.866840 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.866956 (+ 116us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.866996 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 18:48:23.867273 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.862445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41711 (local address 127.15.77.65:48371)
0620 18:48:23.862652 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.862686 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:23.863272 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:23.863546 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:23.863556 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:23.863565 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:23.864057 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.864062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.864795 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.864796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.865586 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.865588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.866006 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.866007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.866140 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.866616 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:48:23.866617 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:23.866644 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:48:23.866666 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:23.866847 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:23.866852 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:23.866856 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:48:23.866957 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 18:48:23.867055 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260620 18:48:23.867791 3161 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41711
I20260620 18:48:23.867898 3161 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:23.868084 3161 heartbeater.cc:511] Master 127.15.77.126:41711 requested a full tablet report, sending...
I20260620 18:48:23.868644 2973 ts_manager.cc:194] Registered new tserver with Master: 90babd83a3e940dabdd1c54f0ba8505d (127.15.77.65:35453)
I20260620 18:48:23.869256 2973 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.65:48371
I20260620 18:48:23.870966 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 3033
I20260620 18:48:23.871042 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-0/wal/instance
I20260620 18:48:23.871191 15669 external_mini_cluster.cc:1428] Setting key 34e43db1d6839ffb9c8dc4e969d6175f
I20260620 18:48:23.872051 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.66:0
--local_ip_for_outbound_sockets=127.15.77.66
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41711
--builtin_ntp_servers=127.15.77.84:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 18:48:23.952235 3164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:23.952435 3164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:23.952457 3164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:23.953538 3164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:23.953570 3164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:23.953594 3164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.66
I20260620 18:48:23.955518 3164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.77.66
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.66
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:23.955705 3164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:23.955976 3164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:23.960897 3170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:23.960887 3169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:23.960958 3172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:23.960999 3164 server_base.cc:1034] running on GCE node
I20260620 18:48:23.961339 3164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:23.961638 3164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:23.962797 3164 hybrid_clock.cc:648] HybridClock initialized: now 1781981303962788 us; error 37 us; skew 500 ppm
I20260620 18:48:23.964053 3164 webserver.cc:460] Webserver started at http://127.15.77.66:46689/ using document root <none> and password file <none>
I20260620 18:48:23.964294 3164 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:23.964363 3164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:23.964514 3164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:23.965577 3164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/data/instance:
uuid: "3247eaa26088454ba856bbe1afaf982b"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "a0e6c690d1caf337cb5646ced2dc56e8"
server_key_iv: "28d20d906c4b6f552975212ffc35289a"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.965975 3164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance:
uuid: "3247eaa26088454ba856bbe1afaf982b"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "a0e6c690d1caf337cb5646ced2dc56e8"
server_key_iv: "28d20d906c4b6f552975212ffc35289a"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.967293 3164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:23.968112 3178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.968291 3164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:23.968389 3164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/wal
uuid: "3247eaa26088454ba856bbe1afaf982b"
format_stamp: "Formatted at 2026-06-20 18:48:23 on dist-test-slave-tvl9"
server_key: "a0e6c690d1caf337cb5646ced2dc56e8"
server_key_iv: "28d20d906c4b6f552975212ffc35289a"
server_key_version: "encryptionkey@0"
I20260620 18:48:23.968494 3164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:23.985581 3164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:23.985896 3164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:23.986064 3164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:23.986317 3164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:23.986712 3164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:23.986758 3164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.986783 3164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:23.986794 3164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:23.992508 3164 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.66:39509
I20260620 18:48:23.992554 3291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.66:39509 every 8 connection(s)
I20260620 18:48:23.992893 3164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/data/info.pb
I20260620 18:48:23.996709 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 3164
I20260620 18:48:23.996811 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/wal/instance
I20260620 18:48:23.997006 15669 external_mini_cluster.cc:1428] Setting key 8accecbafbe0d91de17c6ce4f8f67cc2
I20260620 18:48:23.998114 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.77.67:0
--local_ip_for_outbound_sockets=127.15.77.67
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41711
--builtin_ntp_servers=127.15.77.84:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 18:48:23.998332 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.993628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.66:57861 (local address 127.15.77.126:41711)
0620 18:48:23.993698 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:23.993700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:23.993966 (+ 266us) server_negotiation.cc:407] Connection header received
0620 18:48:23.994450 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:23.994451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:23.994461 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:23.994521 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:23.995160 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.995726 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.996675 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.996987 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.997063 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.998013 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:23.998014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:23.998015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:23.998020 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:23.998037 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:23.998042 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:23.998111 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:23.998182 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 18:48:23.998215 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 18:48:23.998540 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:23.993621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41711 (local address 127.15.77.66:57861)
0620 18:48:23.993832 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:23.993867 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:23.994351 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:23.994576 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:23.994579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:23.994587 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:23.995088 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.995092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.995810 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.995811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.996587 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:23.996589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:23.997095 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:23.997096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:23.997254 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:23.997849 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:48:23.997849 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:23.997895 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:48:23.997917 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:23.998109 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:23.998111 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:23.998114 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 18:48:23.998158 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 18:48:23.998296 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260620 18:48:23.999023 3292 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41711
I20260620 18:48:23.999125 3292 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:23.999302 3292 heartbeater.cc:511] Master 127.15.77.126:41711 requested a full tablet report, sending...
I20260620 18:48:23.999752 2973 ts_manager.cc:194] Registered new tserver with Master: 3247eaa26088454ba856bbe1afaf982b (127.15.77.66:39509)
I20260620 18:48:24.000123 2973 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.66:57861
W20260620 18:48:24.080188 3295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:24.080439 3295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:24.080476 3295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:24.081781 3295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:24.081880 3295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:24.081931 3295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.77.67
I20260620 18:48:24.083519 3295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.77.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.77.67
--webserver_port=0
--tserver_master_addrs=127.15.77.126:41711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.77.67
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:24.083804 3295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:24.084123 3295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:24.089507 3300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:24.089589 3295 server_base.cc:1034] running on GCE node
W20260620 18:48:24.089512 3301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:24.089589 3303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:24.089993 3295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:24.090258 3295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:24.091377 3295 hybrid_clock.cc:648] HybridClock initialized: now 1781981304091372 us; error 36 us; skew 500 ppm
I20260620 18:48:24.092638 3295 webserver.cc:460] Webserver started at http://127.15.77.67:37925/ using document root <none> and password file <none>
I20260620 18:48:24.092897 3295 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:24.092978 3295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:24.093142 3295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:24.094213 3295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/data/instance:
uuid: "467888ada8c5495891a6cd635c842eb7"
format_stamp: "Formatted at 2026-06-20 18:48:24 on dist-test-slave-tvl9"
server_key: "b1c67a88b65c2e4f905d30f5598fe433"
server_key_iv: "c8292f770149195f884a832354d41faf"
server_key_version: "encryptionkey@0"
I20260620 18:48:24.094570 3295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance:
uuid: "467888ada8c5495891a6cd635c842eb7"
format_stamp: "Formatted at 2026-06-20 18:48:24 on dist-test-slave-tvl9"
server_key: "b1c67a88b65c2e4f905d30f5598fe433"
server_key_iv: "c8292f770149195f884a832354d41faf"
server_key_version: "encryptionkey@0"
I20260620 18:48:24.095937 3295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:24.096673 3309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:24.096841 3295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:24.096939 3295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/wal
uuid: "467888ada8c5495891a6cd635c842eb7"
format_stamp: "Formatted at 2026-06-20 18:48:24 on dist-test-slave-tvl9"
server_key: "b1c67a88b65c2e4f905d30f5598fe433"
server_key_iv: "c8292f770149195f884a832354d41faf"
server_key_version: "encryptionkey@0"
I20260620 18:48:24.097026 3295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:24.106745 3295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:24.107019 3295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:24.107158 3295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:24.107417 3295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 18:48:24.107729 3295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 18:48:24.107782 3295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:24.107822 3295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 18:48:24.107851 3295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:24.113279 3295 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.67:33089
I20260620 18:48:24.113379 3422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.67:33089 every 8 connection(s)
I20260620 18:48:24.113679 3295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/data/info.pb
I20260620 18:48:24.118820 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.114380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.67:53669 (local address 127.15.77.126:41711)
0620 18:48:24.114469 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:24.114471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:24.114666 (+ 195us) server_negotiation.cc:407] Connection header received
0620 18:48:24.115123 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:24.115124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:24.115133 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:24.115185 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:24.115885 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.116371 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.117355 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.117594 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.117650 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.118605 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:24.118606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:24.118606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:24.118611 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:24.118623 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:24.118627 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:24.118670 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:24.118735 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 18:48:24.118759 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 18:48:24.119005 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.114343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:41711 (local address 127.15.77.67:53669)
0620 18:48:24.114543 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:24.114576 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:24.115044 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:24.115237 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:24.115240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:24.115248 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:24.115806 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.115812 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.116449 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.116450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.117239 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.117241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.117670 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.117671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.117831 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.118469 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 18:48:24.118469 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:24.118500 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 18:48:24.118523 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:24.118670 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:24.118672 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:24.118674 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 18:48:24.118720 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 18:48:24.118799 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260620 18:48:24.119582 3423 heartbeater.cc:347] Connected to a master server at 127.15.77.126:41711
I20260620 18:48:24.119688 3423 heartbeater.cc:464] Registering TS with master...
I20260620 18:48:24.119871 3423 heartbeater.cc:511] Master 127.15.77.126:41711 requested a full tablet report, sending...
I20260620 18:48:24.120265 2973 ts_manager.cc:194] Registered new tserver with Master: 467888ada8c5495891a6cd635c842eb7 (127.15.77.67:33089)
I20260620 18:48:24.120599 2973 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.77.67:53669
I20260620 18:48:24.123286 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 3295
I20260620 18:48:24.123426 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/wal/instance
I20260620 18:48:24.123636 15669 external_mini_cluster.cc:1428] Setting key 9bec50a29c760465ba771adf73a5ce19
I20260620 18:48:24.124504 15669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 18:48:24.128875 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.125848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.15.77.126:41711)
0620 18:48:24.125931 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:24.125933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:24.126033 (+ 100us) server_negotiation.cc:407] Connection header received
0620 18:48:24.126119 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:24.126120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:24.126133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:24.126187 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:24.126355 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.126831 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.127669 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.127860 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.127942 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.128635 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:24.128636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:24.128637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:24.128642 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:24.128656 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:24.128660 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:24.128702 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:24.128767 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 18:48:24.128790 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 18:48:24.130349 2973 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 18:48:24.131927 2973 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 18:48:24.138466 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.134295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35990 (local address 127.15.77.67:33089)
0620 18:48:24.134774 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:24.134779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:24.134788 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:24.134826 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:24.134828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:24.134843 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:24.134922 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:24.135390 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.136053 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.137595 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.138071 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.138158 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.138248 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:24.138338 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 18:48:24.138378 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":105,"threads_started":1}
I20260620 18:48:24.138455 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.134051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:33089 (local address 127.0.0.1:35990)
0620 18:48:24.134490 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:24.134500 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:24.134541 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:24.134995 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:24.134996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:24.134997 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:24.135054 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.135055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.136143 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.136145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.137495 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.137496 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.138158 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.138159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.138236 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.138248 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 18:48:24.138323 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 18:48:24.138377 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":95,"thread_start_us":98,"threads_started":1}
I20260620 18:48:24.138612 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.134051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:35453 (local address 127.0.0.1:39698)
0620 18:48:24.134595 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:24.134601 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:24.134652 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:24.134992 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:24.134993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:24.134995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:24.135055 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.135057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.136311 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.136313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.137287 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.137290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.138287 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.138288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.138352 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.138356 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:48:24.138402 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 18:48:24.138446 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":24,"spinlock_wait_cycles":896,"thread_start_us":92,"threads_started":1}
I20260620 18:48:24.138662 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.134523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39698 (local address 127.15.77.65:35453)
0620 18:48:24.134792 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:24.134796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:24.134805 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:24.134841 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:24.134843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:24.134856 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:24.134926 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:24.135140 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.136226 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.137383 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.138086 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.138159 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.138247 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:24.138504 (+ 257us) server_negotiation.cc:299] Negotiation successful
0620 18:48:24.138550 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":97,"threads_started":1}
I20260620 18:48:24.139256 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.134019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:39509 (local address 127.0.0.1:41450)
0620 18:48:24.134130 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:24.134167 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:24.134250 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:24.134678 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:24.134682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:24.134684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:24.134748 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.134749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.136200 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.136201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.137493 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.137496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.138449 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.138450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.138504 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.138508 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 18:48:24.139133 (+ 625us) client_negotiation.cc:240] Negotiation successful
0620 18:48:24.139177 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":30,"thread_start_us":67,"threads_started":1}
I20260620 18:48:24.139444 3095 tablet_service.cc:1467] Processing CreateTablet for tablet 23ea3fa476784144924bd467b0ad38a1 (DEFAULT_TABLE table=test-table [id=9ce062475eb24b78a59530e098ef6c20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:24.139430 3357 tablet_service.cc:1467] Processing CreateTablet for tablet 23ea3fa476784144924bd467b0ad38a1 (DEFAULT_TABLE table=test-table [id=9ce062475eb24b78a59530e098ef6c20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:24.139580 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.134309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41450 (local address 127.15.77.66:39509)
0620 18:48:24.134486 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:24.134491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:24.134503 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:48:24.134544 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:24.134546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:24.134564 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:24.134653 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:24.134891 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.136095 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.137587 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.138280 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.138341 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.138417 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:24.139461 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0620 18:48:24.139503 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260620 18:48:24.139827 3357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ea3fa476784144924bd467b0ad38a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:24.139830 3095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ea3fa476784144924bd467b0ad38a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:24.140179 3226 tablet_service.cc:1467] Processing CreateTablet for tablet 23ea3fa476784144924bd467b0ad38a1 (DEFAULT_TABLE table=test-table [id=9ce062475eb24b78a59530e098ef6c20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 18:48:24.140508 3226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ea3fa476784144924bd467b0ad38a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:24.142449 3437 tablet_bootstrap.cc:492] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: Bootstrap starting.
I20260620 18:48:24.143211 3437 tablet_bootstrap.cc:654] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:24.143203 3438 tablet_bootstrap.cc:492] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d: Bootstrap starting.
I20260620 18:48:24.143654 3437 log.cc:824] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:24.143743 3439 tablet_bootstrap.cc:492] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: Bootstrap starting.
I20260620 18:48:24.143875 3438 tablet_bootstrap.cc:654] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:24.144194 3438 log.cc:824] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:24.144346 3439 tablet_bootstrap.cc:654] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:24.144649 3439 log.cc:824] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:24.144866 3437 tablet_bootstrap.cc:492] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: No bootstrap required, opened a new log
I20260620 18:48:24.144956 3437 ts_tablet_manager.cc:1397] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 18:48:24.145285 3439 tablet_bootstrap.cc:492] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: No bootstrap required, opened a new log
I20260620 18:48:24.145335 3439 ts_tablet_manager.cc:1397] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 18:48:24.147123 3437 raft_consensus.cc:348] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 } } peers { permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 } } peers { permanent_uuid: "90babd83a3e940dabdd1c54f0ba8505d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35453 } }
I20260620 18:48:24.147284 3437 raft_consensus.cc:374] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:24.147324 3437 raft_consensus.cc:729] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 467888ada8c5495891a6cd635c842eb7, State: Initialized, Role: FOLLOWER
I20260620 18:48:24.147503 3437 consensus_queue.cc:260] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 } } peers { permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 } } peers { permanent_uuid: "90babd83a3e940dabdd1c54f0ba8505d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35453 } }
I20260620 18:48:24.147544 3439 raft_consensus.cc:348] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 } } peers { permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 } } peers { permanent_uuid: "90babd83a3e940dabdd1c54f0ba8505d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35453 } }
I20260620 18:48:24.147717 3439 raft_consensus.cc:374] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:24.147761 3439 raft_consensus.cc:729] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3247eaa26088454ba856bbe1afaf982b, State: Initialized, Role: FOLLOWER
I20260620 18:48:24.147815 3437 ts_tablet_manager.cc:1428] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260620 18:48:24.147851 3423 heartbeater.cc:503] Master 127.15.77.126:41711 was elected leader, sending a full tablet report...
I20260620 18:48:24.147883 3439 consensus_queue.cc:260] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 } } peers { permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 } } peers { permanent_uuid: "90babd83a3e940dabdd1c54f0ba8505d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35453 } }
I20260620 18:48:24.148135 3439 ts_tablet_manager.cc:1428] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 18:48:24.148170 3292 heartbeater.cc:503] Master 127.15.77.126:41711 was elected leader, sending a full tablet report...
I20260620 18:48:24.148799 3438 tablet_bootstrap.cc:492] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d: No bootstrap required, opened a new log
I20260620 18:48:24.148886 3438 ts_tablet_manager.cc:1397] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260620 18:48:24.150820 3438 raft_consensus.cc:348] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 } } peers { permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 } } peers { permanent_uuid: "90babd83a3e940dabdd1c54f0ba8505d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35453 } }
I20260620 18:48:24.150951 3438 raft_consensus.cc:374] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:24.150975 3438 raft_consensus.cc:729] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90babd83a3e940dabdd1c54f0ba8505d, State: Initialized, Role: FOLLOWER
I20260620 18:48:24.151085 3438 consensus_queue.cc:260] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 } } peers { permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 } } peers { permanent_uuid: "90babd83a3e940dabdd1c54f0ba8505d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35453 } }
I20260620 18:48:24.151295 3438 ts_tablet_manager.cc:1428] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 18:48:24.151391 3161 heartbeater.cc:503] Master 127.15.77.126:41711 was elected leader, sending a full tablet report...
W20260620 18:48:24.243515 3293 tablet.cc:2369] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:48:24.337778 3445 raft_consensus.cc:482] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 18:48:24.337971 3445 raft_consensus.cc:504] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 } } peers { permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 } } peers { permanent_uuid: "90babd83a3e940dabdd1c54f0ba8505d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35453 } }
I20260620 18:48:24.338390 3445 leader_election.cc:288] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3247eaa26088454ba856bbe1afaf982b (127.15.77.66:39509), 467888ada8c5495891a6cd635c842eb7 (127.15.77.67:33089)
I20260620 18:48:24.341360 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.338692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:43553 (local address 127.15.77.66:39509)
0620 18:48:24.338770 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:24.338772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:24.338782 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:48:24.338814 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:24.338815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:24.338825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:24.338889 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:24.339093 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.339676 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.340643 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.341005 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.341061 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.341067 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:24.341240 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 18:48:24.341286 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 18:48:24.341387 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.338598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.66:39509 (local address 127.15.77.65:43553)
0620 18:48:24.338660 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:24.338671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:24.338730 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:24.338922 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:24.338923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:24.338925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:24.339008 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.339011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.339769 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.339770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.340548 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.340551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.341087 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.341088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.341167 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.341175 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 18:48:24.341216 (+ 41us) client_negotiation.cc:240] Negotiation successful
0620 18:48:24.341247 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":3}
I20260620 18:48:24.341428 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.338817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.77.65:50643 (local address 127.15.77.67:33089)
0620 18:48:24.338861 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:24.338863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:24.338922 (+ 59us) server_negotiation.cc:407] Connection header received
0620 18:48:24.338957 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:24.338958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:24.338967 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:24.339035 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 18:48:24.339212 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.339738 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.340680 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.341023 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.341118 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.341125 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:24.341308 (+ 183us) server_negotiation.cc:299] Negotiation successful
0620 18:48:24.341353 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 18:48:24.341498 3446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.338684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.67:33089 (local address 127.15.77.65:50643)
0620 18:48:24.338852 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:24.338861 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:24.338909 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:24.339065 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:24.339066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:24.339068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 18:48:24.339139 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.339141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.339824 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.339825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.340563 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:24.340564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.341141 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.341141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:24.341225 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.341230 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 18:48:24.341286 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 18:48:24.341320 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":1,"thread_start_us":101,"threads_started":1}
I20260620 18:48:24.341781 3377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ea3fa476784144924bd467b0ad38a1" candidate_uuid: "90babd83a3e940dabdd1c54f0ba8505d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "467888ada8c5495891a6cd635c842eb7" is_pre_election: true
I20260620 18:48:24.341785 3246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ea3fa476784144924bd467b0ad38a1" candidate_uuid: "90babd83a3e940dabdd1c54f0ba8505d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3247eaa26088454ba856bbe1afaf982b" is_pre_election: true
I20260620 18:48:24.341948 3377 raft_consensus.cc:2454] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90babd83a3e940dabdd1c54f0ba8505d in term 0.
I20260620 18:48:24.341971 3246 raft_consensus.cc:2454] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90babd83a3e940dabdd1c54f0ba8505d in term 0.
I20260620 18:48:24.342195 3048 leader_election.cc:302] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 467888ada8c5495891a6cd635c842eb7, 90babd83a3e940dabdd1c54f0ba8505d; no voters:
I20260620 18:48:24.342370 3445 raft_consensus.cc:2781] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 18:48:24.342407 3445 raft_consensus.cc:482] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 18:48:24.342456 3445 raft_consensus.cc:3037] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:24.343154 3445 raft_consensus.cc:504] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 } } peers { permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 } } peers { permanent_uuid: "90babd83a3e940dabdd1c54f0ba8505d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35453 } }
I20260620 18:48:24.343413 3445 leader_election.cc:288] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [CANDIDATE]: Term 1 election: Requested vote from peers 3247eaa26088454ba856bbe1afaf982b (127.15.77.66:39509), 467888ada8c5495891a6cd635c842eb7 (127.15.77.67:33089)
I20260620 18:48:24.343497 3246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ea3fa476784144924bd467b0ad38a1" candidate_uuid: "90babd83a3e940dabdd1c54f0ba8505d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3247eaa26088454ba856bbe1afaf982b"
I20260620 18:48:24.343511 3377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ea3fa476784144924bd467b0ad38a1" candidate_uuid: "90babd83a3e940dabdd1c54f0ba8505d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "467888ada8c5495891a6cd635c842eb7"
I20260620 18:48:24.343572 3377 raft_consensus.cc:3037] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:24.343572 3246 raft_consensus.cc:3037] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:24.344463 3377 raft_consensus.cc:2454] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90babd83a3e940dabdd1c54f0ba8505d in term 1.
I20260620 18:48:24.344463 3246 raft_consensus.cc:2454] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90babd83a3e940dabdd1c54f0ba8505d in term 1.
I20260620 18:48:24.344650 3050 leader_election.cc:302] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3247eaa26088454ba856bbe1afaf982b, 90babd83a3e940dabdd1c54f0ba8505d; no voters:
I20260620 18:48:24.344748 3445 raft_consensus.cc:2781] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:24.344894 3445 raft_consensus.cc:686] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [term 1 LEADER]: Becoming Leader. State: Replica: 90babd83a3e940dabdd1c54f0ba8505d, State: Running, Role: LEADER
I20260620 18:48:24.345016 3445 consensus_queue.cc:237] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 } } peers { permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 } } peers { permanent_uuid: "90babd83a3e940dabdd1c54f0ba8505d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35453 } }
I20260620 18:48:24.345901 2972 catalog_manager.cc:5495] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d reported cstate change: term changed from 0 to 1, leader changed from <none> to 90babd83a3e940dabdd1c54f0ba8505d (127.15.77.65). New cstate: current_term: 1 leader_uuid: "90babd83a3e940dabdd1c54f0ba8505d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90babd83a3e940dabdd1c54f0ba8505d" member_type: VOTER last_known_addr { host: "127.15.77.65" port: 35453 } health_report { overall_health: HEALTHY } } }
W20260620 18:48:24.362475 3162 tablet.cc:2369] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 18:48:24.364202 3424 tablet.cc:2369] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 18:48:24.368847 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.366000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39706 (local address 127.15.77.65:35453)
0620 18:48:24.366042 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:24.366044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:24.366063 (+ 19us) server_negotiation.cc:407] Connection header received
0620 18:48:24.366097 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:24.366098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:24.366116 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:24.366190 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:24.366282 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.366755 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.368080 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:24.368269 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:24.368324 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:24.368479 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:24.368622 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:24.368717 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:24.368757 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 18:48:24.368776 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:48:24.370620 3246 raft_consensus.cc:1261] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b [term 1 FOLLOWER]: Refusing update from remote peer 90babd83a3e940dabdd1c54f0ba8505d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:24.370620 3377 raft_consensus.cc:1261] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7 [term 1 FOLLOWER]: Refusing update from remote peer 90babd83a3e940dabdd1c54f0ba8505d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:24.370952 3445 consensus_queue.cc:1035] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3247eaa26088454ba856bbe1afaf982b" member_type: VOTER last_known_addr { host: "127.15.77.66" port: 39509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:24.371759 3445 consensus_queue.cc:1035] T 23ea3fa476784144924bd467b0ad38a1 P 90babd83a3e940dabdd1c54f0ba8505d [LEADER]: Connected to new peer: Peer: permanent_uuid: "467888ada8c5495891a6cd635c842eb7" member_type: VOTER last_known_addr { host: "127.15.77.67" port: 33089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:24.373328 3450 mvcc.cc:205] Tried to move back new op lower bound from 7298995422699491328 to 7298995422597890048. Current Snapshot: MvccSnapshot[applied={T|T < 7298995422699491328}]
I20260620 18:48:24.373603 3451 mvcc.cc:205] Tried to move back new op lower bound from 7298995422699491328 to 7298995422597890048. Current Snapshot: MvccSnapshot[applied={T|T < 7298995422699491328}]
I20260620 18:48:24.375012 3453 mvcc.cc:205] Tried to move back new op lower bound from 7298995422699491328 to 7298995422597890048. Current Snapshot: MvccSnapshot[applied={T|T < 7298995422699491328}]
I20260620 18:48:24.376714 2972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:24.376816 2972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 18:48:24.377641 2972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb43b8a2722e467ca19d0bdab2eaa33e: Sending DeleteTablet for 3 replicas of tablet 23ea3fa476784144924bd467b0ad38a1
I20260620 18:48:24.378013 3226 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ea3fa476784144924bd467b0ad38a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:24 UTC) from {username='slave'} at 127.0.0.1:41450
I20260620 18:48:24.378069 3357 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ea3fa476784144924bd467b0ad38a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:24 UTC) from {username='slave'} at 127.0.0.1:35990
I20260620 18:48:24.378073 3095 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ea3fa476784144924bd467b0ad38a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 18:48:24 UTC) from {username='slave'} at 127.0.0.1:39698
I20260620 18:48:24.378328 3461 tablet_replica.cc:330] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: stopping tablet replica
I20260620 18:48:24.378433 3461 raft_consensus.cc:2229] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:48:24.378537 3461 raft_consensus.cc:2258] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:24.378604 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 3033
I20260620 18:48:24.378691 3460 tablet_replica.cc:330] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: stopping tablet replica
I20260620 18:48:24.378784 3460 raft_consensus.cc:2229] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 18:48:24.378875 3460 raft_consensus.cc:2258] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 18:48:24.378995 3461 ts_tablet_manager.cc:1905] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:24.379676 3460 ts_tablet_manager.cc:1905] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 18:48:24.381071 3461 ts_tablet_manager.cc:1918] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:24.381139 3461 log.cc:1196] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-2/wal/wals/23ea3fa476784144924bd467b0ad38a1
I20260620 18:48:24.381409 3461 ts_tablet_manager.cc:1939] T 23ea3fa476784144924bd467b0ad38a1 P 467888ada8c5495891a6cd635c842eb7: Deleting consensus metadata
I20260620 18:48:24.381619 3460 ts_tablet_manager.cc:1918] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 18:48:24.381695 3460 log.cc:1196] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: Deleting WAL directory at /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781981032911509-15669-0/minicluster-data/ts-1/wal/wals/23ea3fa476784144924bd467b0ad38a1
I20260620 18:48:24.381867 2957 catalog_manager.cc:4841] TS 467888ada8c5495891a6cd635c842eb7 (127.15.77.67:33089): tablet 23ea3fa476784144924bd467b0ad38a1 (table test-table [id=9ce062475eb24b78a59530e098ef6c20]) successfully deleted
I20260620 18:48:24.381975 3460 ts_tablet_manager.cc:1939] T 23ea3fa476784144924bd467b0ad38a1 P 3247eaa26088454ba856bbe1afaf982b: Deleting consensus metadata
I20260620 18:48:24.382414 2959 catalog_manager.cc:4841] TS 3247eaa26088454ba856bbe1afaf982b (127.15.77.66:39509): tablet 23ea3fa476784144924bd467b0ad38a1 (table test-table [id=9ce062475eb24b78a59530e098ef6c20]) successfully deleted
W20260620 18:48:24.385843 2960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.77.65:35453 (error 108)
I20260620 18:48:24.386050 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 3164
I20260620 18:48:24.387027 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:24.386860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.65:35453 (local address 127.0.0.1:39714)
0620 18:48:24.386934 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:24.386984 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.65:35453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260620 18:48:24.387199 2960 catalog_manager.cc:4584] TS 90babd83a3e940dabdd1c54f0ba8505d (127.15.77.65:35453): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23ea3fa476784144924bd467b0ad38a1: Network error: Client connection negotiation failed: client connection to 127.15.77.65:35453: connect: Connection refused (error 111)
I20260620 18:48:24.390889 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 3295
I20260620 18:48:24.395347 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 2942
2026-06-20T18:48:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (851 ms)
[----------] 6 tests from EncryptionPolicyTest (8401 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:24 dist-test-slave-tvl9 krb5kdc[3466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:24 dist-test-slave-tvl9 krb5kdc[3466](info): set up 2 sockets
Jun 20 18:48:24 dist-test-slave-tvl9 krb5kdc[3466](info): commencing operation
krb5kdc: starting...
W20260620 18:48:25.431241 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:25 dist-test-slave-tvl9 krb5kdc[3466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:25.549629 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:25.631595 3479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:25.631831 3479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:25.631857 3479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:25.632994 3479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:25.633031 3479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:25.633046 3479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:25.633059 3479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:25.633071 3479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:25.634521 3479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42293
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:25.634737 3479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:25.634919 3479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:25.640012 3487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:25.640015 3485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:25.640018 3484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:25.640415 3479 server_base.cc:1034] running on GCE node
I20260620 18:48:25.640705 3479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:25.641044 3479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:25.642220 3479 hybrid_clock.cc:648] HybridClock initialized: now 1781981305642210 us; error 44 us; skew 500 ppm
Jun 20 18:48:25 dist-test-slave-tvl9 krb5kdc[3466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981305, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:25.644972 3479 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:25.645503 3479 webserver.cc:460] Webserver started at http://127.0.0.1:35291/ using document root <none> and password file <none>
I20260620 18:48:25.645756 3479 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:25.645835 3479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:25.646009 3479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:25.647119 3479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "df4e437ca7b24c998c777a779d53e028"
format_stamp: "Formatted at 2026-06-20 18:48:25 on dist-test-slave-tvl9"
server_key: "4ee7df34881c81d86ff07a52de00b56e"
server_key_iv: "ce9d2a46f4b29face2bfaf813b6af93a"
server_key_version: "encryptionkey@0"
I20260620 18:48:25.647526 3479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "df4e437ca7b24c998c777a779d53e028"
format_stamp: "Formatted at 2026-06-20 18:48:25 on dist-test-slave-tvl9"
server_key: "4ee7df34881c81d86ff07a52de00b56e"
server_key_iv: "ce9d2a46f4b29face2bfaf813b6af93a"
server_key_version: "encryptionkey@0"
I20260620 18:48:25.649031 3479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:25.649893 3494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:25.650107 3479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:25.650225 3479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "df4e437ca7b24c998c777a779d53e028"
format_stamp: "Formatted at 2026-06-20 18:48:25 on dist-test-slave-tvl9"
server_key: "4ee7df34881c81d86ff07a52de00b56e"
server_key_iv: "ce9d2a46f4b29face2bfaf813b6af93a"
server_key_version: "encryptionkey@0"
I20260620 18:48:25.650311 3479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:25.667039 3479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:25.669775 3479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:25.669976 3479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:25.674479 3479 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42293
I20260620 18:48:25.674621 3546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42293 every 8 connection(s)
I20260620 18:48:25.674949 3479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:25.675921 3547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:25.678877 3547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028: Bootstrap starting.
I20260620 18:48:25.679659 3547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:25.679982 3547 log.cc:824] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:25.680776 3547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028: No bootstrap required, opened a new log
I20260620 18:48:25.682427 3547 raft_consensus.cc:348] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4e437ca7b24c998c777a779d53e028" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260620 18:48:25.682577 3547 raft_consensus.cc:374] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:25.682601 3547 raft_consensus.cc:729] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df4e437ca7b24c998c777a779d53e028, State: Initialized, Role: FOLLOWER
I20260620 18:48:25.682758 3547 consensus_queue.cc:260] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4e437ca7b24c998c777a779d53e028" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260620 18:48:25.682837 3547 raft_consensus.cc:388] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:25.682878 3547 raft_consensus.cc:482] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:25.682927 3547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:25.683521 3547 raft_consensus.cc:504] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4e437ca7b24c998c777a779d53e028" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260620 18:48:25.683635 3547 leader_election.cc:302] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df4e437ca7b24c998c777a779d53e028; no voters:
I20260620 18:48:25.683830 3547 leader_election.cc:288] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:25.683910 3550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:25.684121 3550 raft_consensus.cc:686] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [term 1 LEADER]: Becoming Leader. State: Replica: df4e437ca7b24c998c777a779d53e028, State: Running, Role: LEADER
I20260620 18:48:25.684131 3547 sys_catalog.cc:564] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:25.684264 3550 consensus_queue.cc:237] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4e437ca7b24c998c777a779d53e028" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260620 18:48:25.684808 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 3479
I20260620 18:48:25.684834 3551 sys_catalog.cc:455] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df4e437ca7b24c998c777a779d53e028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4e437ca7b24c998c777a779d53e028" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } } }
I20260620 18:48:25.684917 3551 sys_catalog.cc:458] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:25.684933 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:25.685135 15669 external_mini_cluster.cc:1428] Setting key 64cdf51ea236abf245da5078f42a9f44
I20260620 18:48:25.685381 3552 sys_catalog.cc:455] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df4e437ca7b24c998c777a779d53e028. Latest consensus state: current_term: 1 leader_uuid: "df4e437ca7b24c998c777a779d53e028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4e437ca7b24c998c777a779d53e028" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } } }
I20260620 18:48:25.685755 3552 sys_catalog.cc:458] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028 [sys.catalog]: This master's current role is: LEADER
W20260620 18:48:25.686358 3566 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:25.686414 3566 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:25.686465 3559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:25.686650 3559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:25.688184 3559 catalog_manager.cc:1347] Generated new cluster ID: 43972c78bba44069916ab06597510b90
I20260620 18:48:25.688241 3559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:48:25 dist-test-slave-tvl9 krb5kdc[3466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 18:48:25.697201 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:25.685646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44427 (local address 127.0.0.1:42293)
0620 18:48:25.686016 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:25.686030 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:25.686054 (+ 24us) server_negotiation.cc:407] Connection header received
0620 18:48:25.686151 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:25.686156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:25.686199 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:25.686337 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:25.687101 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:25.688506 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:25.689266 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:25.689692 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:25.689828 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:25.692348 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:25.692354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:25.692355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:25.692376 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:25.694725 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:25.695233 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:25.695236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:25.695237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:25.695401 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:25.695682 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:25.695683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:25.695684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:25.696055 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:25.696712 (+ 657us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:25.696814 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 18:48:25.696935 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":63,"threads_started":1}
I20260620 18:48:25.698204 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:25.710453 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:25.703764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54025 (local address 127.0.0.1:42293)
0620 18:48:25.703811 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:25.703813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:25.704050 (+ 237us) server_negotiation.cc:407] Connection header received
0620 18:48:25.704137 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:25.704138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:25.704149 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:25.704207 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:25.704816 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:25.705330 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:25.706073 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:25.706297 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:25.706378 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:25.707737 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:25.707739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:25.707739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:25.707747 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:25.709212 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:25.709611 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:25.709613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:25.709613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:25.709672 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:25.709896 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:25.709897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:25.709898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:25.710013 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:25.710148 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:25.710260 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 18:48:25.710326 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:48:25.710919 3559 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:25.711661 3559 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:25.716792 3559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df4e437ca7b24c998c777a779d53e028: Generated new TSK 0
I20260620 18:48:25.717036 3559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:25.721517 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 3479
W20260620 18:48:25.725971 3464 connection.cc:535] client connection to 127.0.0.1:42293 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42293): Connection reset by peer (error 104)
2026-06-20T18:48:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:25 dist-test-slave-tvl9 krb5kdc[3578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:25 dist-test-slave-tvl9 krb5kdc[3578](info): set up 2 sockets
Jun 20 18:48:25 dist-test-slave-tvl9 krb5kdc[3578](info): commencing operation
krb5kdc: starting...
W20260620 18:48:26.759500 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:26 dist-test-slave-tvl9 krb5kdc[3578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:26.877861 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40355
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:26.959854 3591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:26.960106 3591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:26.960142 3591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:26.961313 3591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:26.961390 3591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:26.961416 3591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:26.961437 3591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:26.961467 3591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:26.962971 3591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40355
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:26.963268 3591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:26.963532 3591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:26.968308 3599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:26.968319 3597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:26.968541 3591 server_base.cc:1034] running on GCE node
W20260620 18:48:26.968315 3596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:26.968878 3591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:26.969132 3591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:26.970279 3591 hybrid_clock.cc:648] HybridClock initialized: now 1781981306970269 us; error 34 us; skew 500 ppm
Jun 20 18:48:26 dist-test-slave-tvl9 krb5kdc[3578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981306, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:26.977098 3591 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:26.977631 3591 webserver.cc:460] Webserver started at http://127.0.0.1:43673/ using document root <none> and password file <none>
I20260620 18:48:26.977825 3591 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:26.977867 3591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:26.977962 3591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:26.979022 3591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "6e195b5360c84a8ea840ee86ec04795b"
format_stamp: "Formatted at 2026-06-20 18:48:26 on dist-test-slave-tvl9"
server_key: "e3dddfacf1f86c42022b1c77ca688c51"
server_key_iv: "e8cd387354786ea286d657894797aa42"
server_key_version: "encryptionkey@0"
I20260620 18:48:26.979413 3591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "6e195b5360c84a8ea840ee86ec04795b"
format_stamp: "Formatted at 2026-06-20 18:48:26 on dist-test-slave-tvl9"
server_key: "e3dddfacf1f86c42022b1c77ca688c51"
server_key_iv: "e8cd387354786ea286d657894797aa42"
server_key_version: "encryptionkey@0"
I20260620 18:48:26.980809 3591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:26.981653 3606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:26.981874 3591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:26.981988 3591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "6e195b5360c84a8ea840ee86ec04795b"
format_stamp: "Formatted at 2026-06-20 18:48:26 on dist-test-slave-tvl9"
server_key: "e3dddfacf1f86c42022b1c77ca688c51"
server_key_iv: "e8cd387354786ea286d657894797aa42"
server_key_version: "encryptionkey@0"
I20260620 18:48:26.982069 3591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:27.009845 3591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:27.012509 3591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:27.012704 3591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:27.017007 3591 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40355
I20260620 18:48:27.017047 3658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40355 every 8 connection(s)
I20260620 18:48:27.017414 3591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:27.018296 3659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:27.020780 3659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b: Bootstrap starting.
I20260620 18:48:27.021396 3659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:27.021701 3659 log.cc:824] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:27.022416 3659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b: No bootstrap required, opened a new log
I20260620 18:48:27.023063 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 3591
I20260620 18:48:27.023211 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:27.023443 15669 external_mini_cluster.cc:1428] Setting key c9f7f586dbd246682801365de042a67b
I20260620 18:48:27.024688 3659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e195b5360c84a8ea840ee86ec04795b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40355 } }
I20260620 18:48:27.024870 3659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:27.024914 3659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e195b5360c84a8ea840ee86ec04795b, State: Initialized, Role: FOLLOWER
I20260620 18:48:27.025053 3659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e195b5360c84a8ea840ee86ec04795b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40355 } }
I20260620 18:48:27.025136 3659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:27.025161 3659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:27.025223 3659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:27.025813 3659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e195b5360c84a8ea840ee86ec04795b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40355 } }
I20260620 18:48:27.025899 3659 leader_election.cc:302] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e195b5360c84a8ea840ee86ec04795b; no voters:
I20260620 18:48:27.026077 3659 leader_election.cc:288] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:27.026101 3664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:27.026206 3664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [term 1 LEADER]: Becoming Leader. State: Replica: 6e195b5360c84a8ea840ee86ec04795b, State: Running, Role: LEADER
I20260620 18:48:27.026288 3664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e195b5360c84a8ea840ee86ec04795b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40355 } }
I20260620 18:48:27.026348 3659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:27.026599 3664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e195b5360c84a8ea840ee86ec04795b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e195b5360c84a8ea840ee86ec04795b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40355 } } }
I20260620 18:48:27.026670 3664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:27.026798 3664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e195b5360c84a8ea840ee86ec04795b. Latest consensus state: current_term: 1 leader_uuid: "6e195b5360c84a8ea840ee86ec04795b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e195b5360c84a8ea840ee86ec04795b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40355 } } }
I20260620 18:48:27.026865 3664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b [sys.catalog]: This master's current role is: LEADER
W20260620 18:48:27.028304 3680 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:27.028375 3680 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:27.028439 3668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:27.028617 3668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:27.030066 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:27.024096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51349 (local address 127.0.0.1:40355)
0620 18:48:27.024407 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:27.024416 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:27.024453 (+ 37us) server_negotiation.cc:407] Connection header received
0620 18:48:27.024641 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:27.024647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:27.024690 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:27.024806 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:27.025052 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:27.026642 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:27.027828 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:27.028472 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:27.028578 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:27.029344 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:27.029348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:27.029348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:27.029560 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:27.029604 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:27.029619 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:27.029708 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:27.029811 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 18:48:27.029844 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"spinlock_wait_cycles":207104,"thread_start_us":111,"threads_started":1}
I20260620 18:48:27.030290 3668 catalog_manager.cc:1347] Generated new cluster ID: ab8cdb14642c40bb92449baa60c218e4
I20260620 18:48:27.030341 3668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:27.031013 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:27.039500 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:27.035678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37253 (local address 127.0.0.1:40355)
0620 18:48:27.035724 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:27.035726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:27.035892 (+ 166us) server_negotiation.cc:407] Connection header received
0620 18:48:27.035965 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:27.035966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:27.035978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:27.036037 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:27.036212 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:27.036942 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:27.038011 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:27.038308 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:27.038367 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:27.039178 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:27.039180 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:27.039180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:27.039186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:27.039204 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:27.039209 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:27.039271 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:27.039390 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 18:48:27.039404 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 18:48:27.040961 3668 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:27.041460 3668 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:27.046985 3668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e195b5360c84a8ea840ee86ec04795b: Generated new TSK 0
I20260620 18:48:27.047178 3668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:27.053285 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 3591
W20260620 18:48:27.057433 3576 connection.cc:535] client connection to 127.0.0.1:40355 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40355): Connection reset by peer (error 104)
2026-06-20T18:48:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:27 dist-test-slave-tvl9 krb5kdc[3690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:27 dist-test-slave-tvl9 krb5kdc[3690](info): set up 2 sockets
Jun 20 18:48:27 dist-test-slave-tvl9 krb5kdc[3690](info): commencing operation
krb5kdc: starting...
W20260620 18:48:28.099287 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:28 dist-test-slave-tvl9 krb5kdc[3690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:28.222129 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44605
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:28.305754 3703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:28.305966 3703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:28.305990 3703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:28.307269 3703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:28.307328 3703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:28.307344 3703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:28.307386 3703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:28.307415 3703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:28.308972 3703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:28.309247 3703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:28.309486 3703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:28.314692 3711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:28.314726 3708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:28.314708 3703 server_base.cc:1034] running on GCE node
W20260620 18:48:28.314692 3709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:28.315194 3703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:28.315511 3703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:28.316666 3703 hybrid_clock.cc:648] HybridClock initialized: now 1781981308316649 us; error 28 us; skew 500 ppm
Jun 20 18:48:28 dist-test-slave-tvl9 krb5kdc[3690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981308, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:28.319420 3703 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:28.319849 3703 webserver.cc:460] Webserver started at http://127.0.0.1:43917/ using document root <none> and password file <none>
I20260620 18:48:28.320068 3703 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:28.320111 3703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:28.320204 3703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:28.321247 3703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c"
format_stamp: "Formatted at 2026-06-20 18:48:28 on dist-test-slave-tvl9"
server_key: "fa49a3a0fcdffed3e394d28702c12606"
server_key_iv: "20572bce495f9ac2dd7e93c34182aa85"
server_key_version: "encryptionkey@0"
I20260620 18:48:28.321544 3703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c"
format_stamp: "Formatted at 2026-06-20 18:48:28 on dist-test-slave-tvl9"
server_key: "fa49a3a0fcdffed3e394d28702c12606"
server_key_iv: "20572bce495f9ac2dd7e93c34182aa85"
server_key_version: "encryptionkey@0"
I20260620 18:48:28.322841 3703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:28.323626 3718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:28.323793 3703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:28.323884 3703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c"
format_stamp: "Formatted at 2026-06-20 18:48:28 on dist-test-slave-tvl9"
server_key: "fa49a3a0fcdffed3e394d28702c12606"
server_key_iv: "20572bce495f9ac2dd7e93c34182aa85"
server_key_version: "encryptionkey@0"
I20260620 18:48:28.323953 3703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:28.327276 3703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:28.329248 3703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:28.329399 3703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:28.333310 3703 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44605
I20260620 18:48:28.333361 3770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44605 every 8 connection(s)
I20260620 18:48:28.333712 3703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:28.334713 3771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:28.337087 3771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c: Bootstrap starting.
I20260620 18:48:28.337177 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 3703
I20260620 18:48:28.337324 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:28.337528 15669 external_mini_cluster.cc:1428] Setting key d063898ad6f5d4f9c9bef8ad28eb0c2c
I20260620 18:48:28.337849 3771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:28.338235 3771 log.cc:824] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:28.339006 3771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c: No bootstrap required, opened a new log
I20260620 18:48:28.340262 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:28.338067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49025 (local address 127.0.0.1:44605)
0620 18:48:28.338291 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:28.338301 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:28.338334 (+ 33us) server_negotiation.cc:407] Connection header received
0620 18:48:28.338918 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:28.338927 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:28.338962 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:28.339116 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:28.339817 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:28.339822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:28.339823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:28.339828 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:28.339882 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:28.339899 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:28.339991 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:28.340022 (+ 31us) server_negotiation.cc:299] Negotiation successful
0620 18:48:28.340062 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260620 18:48:28.341388 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:28.341617 3771 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44605 } }
I20260620 18:48:28.341784 3771 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:28.341815 3771 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8bb1b7cc4a043f6ba17aa68ccbfe03c, State: Initialized, Role: FOLLOWER
I20260620 18:48:28.341949 3771 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44605 } }
I20260620 18:48:28.342043 3771 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:28.342074 3771 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:28.342123 3771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:28.342995 3771 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44605 } }
I20260620 18:48:28.343112 3771 leader_election.cc:302] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8bb1b7cc4a043f6ba17aa68ccbfe03c; no voters:
I20260620 18:48:28.343276 3771 leader_election.cc:288] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:28.343299 3781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:28.343514 3781 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [term 1 LEADER]: Becoming Leader. State: Replica: d8bb1b7cc4a043f6ba17aa68ccbfe03c, State: Running, Role: LEADER
I20260620 18:48:28.343611 3771 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:28.343604 3781 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44605 } }
I20260620 18:48:28.344972 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:28.343010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57481 (local address 127.0.0.1:44605)
0620 18:48:28.343074 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:28.343076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:28.343739 (+ 663us) server_negotiation.cc:407] Connection header received
0620 18:48:28.343835 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:28.343836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:28.343843 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:28.343881 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:28.344641 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:28.344642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:28.344643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:28.344647 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:28.344661 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:28.344666 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:28.344717 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:28.344854 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 18:48:28.344880 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 18:48:28.344887 3783 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8bb1b7cc4a043f6ba17aa68ccbfe03c. Latest consensus state: current_term: 1 leader_uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44605 } } }
I20260620 18:48:28.345093 3783 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:28.345225 3782 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bb1b7cc4a043f6ba17aa68ccbfe03c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44605 } } }
I20260620 18:48:28.345317 3797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:28.345567 3782 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:28.345965 3797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:28.347573 3797 catalog_manager.cc:1347] Generated new cluster ID: 334075f4782344368d86eb54be8fc7a6
I20260620 18:48:28.347617 3797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:28.363519 3797 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:28.363715 3797 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:28.374454 3797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8bb1b7cc4a043f6ba17aa68ccbfe03c: Generated new TSK 0
I20260620 18:48:28.374686 3797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:28.381210 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 3703
2026-06-20T18:48:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:28 dist-test-slave-tvl9 krb5kdc[3802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:28 dist-test-slave-tvl9 krb5kdc[3802](info): set up 2 sockets
Jun 20 18:48:28 dist-test-slave-tvl9 krb5kdc[3802](info): commencing operation
krb5kdc: starting...
W20260620 18:48:29.415205 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:29 dist-test-slave-tvl9 krb5kdc[3802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:29.534699 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:29.615895 3815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:29.616163 3815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:29.616205 3815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:29.617372 3815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:29.617455 3815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:29.617479 3815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:29.617506 3815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:29.617534 3815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:29.619030 3815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:29.619341 3815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:29.619607 3815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:29.624500 3821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:29.624615 3823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:29.624655 3815 server_base.cc:1034] running on GCE node
W20260620 18:48:29.624615 3820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:29.625020 3815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:29.625368 3815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:29.626541 3815 hybrid_clock.cc:648] HybridClock initialized: now 1781981309626516 us; error 36 us; skew 500 ppm
Jun 20 18:48:29 dist-test-slave-tvl9 krb5kdc[3802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981309, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:29.629094 3815 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:29.629518 3815 webserver.cc:460] Webserver started at http://127.0.0.1:46289/ using document root <none> and password file <none>
I20260620 18:48:29.629766 3815 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:29.629835 3815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:29.629977 3815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:29.631102 3815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "19583939dff44042a6480da97f8910cb"
format_stamp: "Formatted at 2026-06-20 18:48:29 on dist-test-slave-tvl9"
server_key: "76946c833b30c8b26f90d487eb4e8692"
server_key_iv: "83a5e6ae8e0df0ba6bdb066ee311c696"
server_key_version: "encryptionkey@0"
I20260620 18:48:29.631485 3815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "19583939dff44042a6480da97f8910cb"
format_stamp: "Formatted at 2026-06-20 18:48:29 on dist-test-slave-tvl9"
server_key: "76946c833b30c8b26f90d487eb4e8692"
server_key_iv: "83a5e6ae8e0df0ba6bdb066ee311c696"
server_key_version: "encryptionkey@0"
I20260620 18:48:29.632798 3815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:29.633522 3830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:29.633701 3815 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:48:29.633801 3815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "19583939dff44042a6480da97f8910cb"
format_stamp: "Formatted at 2026-06-20 18:48:29 on dist-test-slave-tvl9"
server_key: "76946c833b30c8b26f90d487eb4e8692"
server_key_iv: "83a5e6ae8e0df0ba6bdb066ee311c696"
server_key_version: "encryptionkey@0"
I20260620 18:48:29.633885 3815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:29.657886 3815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:29.660455 3815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:29.660636 3815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:29.664776 3815 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35369
I20260620 18:48:29.664820 3882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35369 every 8 connection(s)
I20260620 18:48:29.665203 3815 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:29.666069 3883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:29.668357 3883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb: Bootstrap starting.
I20260620 18:48:29.668907 3883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:29.669222 3883 log.cc:824] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:29.669497 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 3815
I20260620 18:48:29.669625 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:29.669854 15669 external_mini_cluster.cc:1428] Setting key 5cbe46a9111ae29845bafeadc164acb8
I20260620 18:48:29.670123 3883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb: No bootstrap required, opened a new log
I20260620 18:48:29.672304 3883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19583939dff44042a6480da97f8910cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260620 18:48:29.672482 3883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:29.672523 3883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19583939dff44042a6480da97f8910cb, State: Initialized, Role: FOLLOWER
I20260620 18:48:29.672664 3883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19583939dff44042a6480da97f8910cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260620 18:48:29.672758 3883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:29.672789 3883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:29.672837 3883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:29.673563 3883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19583939dff44042a6480da97f8910cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260620 18:48:29.673671 3883 leader_election.cc:302] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19583939dff44042a6480da97f8910cb; no voters:
I20260620 18:48:29.673813 3883 leader_election.cc:288] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:29.673869 3888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:29.674023 3888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [term 1 LEADER]: Becoming Leader. State: Replica: 19583939dff44042a6480da97f8910cb, State: Running, Role: LEADER
I20260620 18:48:29.674037 3883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:29.675498 3888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19583939dff44042a6480da97f8910cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } }
I20260620 18:48:29.676136 3890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19583939dff44042a6480da97f8910cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19583939dff44042a6480da97f8910cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } } }
I20260620 18:48:29.676323 3890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [sys.catalog]: This master's current role is: LEADER
W20260620 18:48:29.676805 3902 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:29.676864 3902 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:29.676968 3903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:29.677138 3903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:29.677461 3889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19583939dff44042a6480da97f8910cb. Latest consensus state: current_term: 1 leader_uuid: "19583939dff44042a6480da97f8910cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19583939dff44042a6480da97f8910cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35369 } } }
I20260620 18:48:29.677558 3889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb [sys.catalog]: This master's current role is: LEADER
Jun 20 18:48:29 dist-test-slave-tvl9 krb5kdc[3802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 18:48:29.680513 3903 catalog_manager.cc:1347] Generated new cluster ID: ab7f46d21dd2417280698fc84346d759
I20260620 18:48:29.680570 3903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:29.682108 3887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:29.670394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55559 (local address 127.0.0.1:35369)
0620 18:48:29.670607 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:29.670618 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:29.670651 (+ 33us) server_negotiation.cc:407] Connection header received
0620 18:48:29.671505 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:29.671515 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:29.671588 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:29.671739 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:29.672341 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:29.674068 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:29.674968 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:29.675481 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:29.675562 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:29.678390 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:29.678394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:29.678395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:29.678407 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:29.680378 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:29.680814 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:29.680817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:29.680818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:29.680914 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:29.681134 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:29.681135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:29.681136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:29.681378 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:29.681544 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:29.681660 (+ 116us) server_negotiation.cc:299] Negotiation successful
0620 18:48:29.681800 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260620 18:48:29.683056 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:29.696892 3903 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:29.697646 3903 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:29.697939 3887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:29.684462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45179 (local address 127.0.0.1:35369)
0620 18:48:29.684499 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:29.684501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:29.691517 (+ 7016us) server_negotiation.cc:407] Connection header received
0620 18:48:29.691608 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:29.691609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:29.691621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:29.691677 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:29.692234 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:29.692756 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:29.693511 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:29.693760 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:29.693848 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:29.695105 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:29.695107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:29.695107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:29.695114 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:29.696651 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:29.697096 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:29.697097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:29.697098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:29.697168 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:29.697370 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:29.697371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:29.697371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:29.697539 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:29.697645 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:29.697785 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 18:48:29.697876 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":18}
I20260620 18:48:29.703785 3903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19583939dff44042a6480da97f8910cb: Generated new TSK 0
I20260620 18:48:29.703979 3903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:29.710393 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 3815
2026-06-20T18:48:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1330 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:29 dist-test-slave-tvl9 krb5kdc[3914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:29 dist-test-slave-tvl9 krb5kdc[3914](info): set up 2 sockets
Jun 20 18:48:29 dist-test-slave-tvl9 krb5kdc[3914](info): commencing operation
krb5kdc: starting...
W20260620 18:48:30.807549 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:30 dist-test-slave-tvl9 krb5kdc[3914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:30.932430 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37925
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:31.015769 3927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:31.015986 3927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:31.016010 3927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:31.017266 3927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:31.017309 3927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:31.017324 3927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:31.017336 3927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:31.017382 3927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:31.018857 3927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:31.019122 3927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:31.019395 3927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:31.024446 3933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:31.024505 3932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:31.024477 3935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:31.024763 3927 server_base.cc:1034] running on GCE node
I20260620 18:48:31.025048 3927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:31.025358 3927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:31.026546 3927 hybrid_clock.cc:648] HybridClock initialized: now 1781981311026534 us; error 43 us; skew 500 ppm
Jun 20 18:48:31 dist-test-slave-tvl9 krb5kdc[3914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981311, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:31.029290 3927 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:31.029763 3927 webserver.cc:460] Webserver started at http://127.0.0.1:44787/ using document root <none> and password file <none>
I20260620 18:48:31.030005 3927 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:31.030073 3927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:31.030228 3927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:31.031422 3927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "6eb47cfb7f6545ad824c8195c773a438"
format_stamp: "Formatted at 2026-06-20 18:48:31 on dist-test-slave-tvl9"
server_key: "85d2515b80819c2e238bbebe97390673"
server_key_iv: "36dd7cee72424783c81c7183a9efd678"
server_key_version: "encryptionkey@0"
I20260620 18:48:31.031796 3927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "6eb47cfb7f6545ad824c8195c773a438"
format_stamp: "Formatted at 2026-06-20 18:48:31 on dist-test-slave-tvl9"
server_key: "85d2515b80819c2e238bbebe97390673"
server_key_iv: "36dd7cee72424783c81c7183a9efd678"
server_key_version: "encryptionkey@0"
I20260620 18:48:31.033265 3927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:48:31.034140 3942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:31.034354 3927 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:31.034469 3927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "6eb47cfb7f6545ad824c8195c773a438"
format_stamp: "Formatted at 2026-06-20 18:48:31 on dist-test-slave-tvl9"
server_key: "85d2515b80819c2e238bbebe97390673"
server_key_iv: "36dd7cee72424783c81c7183a9efd678"
server_key_version: "encryptionkey@0"
I20260620 18:48:31.034570 3927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:31.048700 3927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:31.051129 3927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:31.051301 3927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:31.055294 3927 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37925
I20260620 18:48:31.055388 3994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37925 every 8 connection(s)
I20260620 18:48:31.055770 3927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:31.056664 3995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:31.057722 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 3927
I20260620 18:48:31.057968 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:31.058154 15669 external_mini_cluster.cc:1428] Setting key aff87b71aaabb60409a19494bd132c59
I20260620 18:48:31.059674 3995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438: Bootstrap starting.
I20260620 18:48:31.060415 3995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:31.060730 3995 log.cc:824] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:31.061489 3995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438: No bootstrap required, opened a new log
I20260620 18:48:31.063707 3995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb47cfb7f6545ad824c8195c773a438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37925 } }
I20260620 18:48:31.063910 3995 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:31.063953 3995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb47cfb7f6545ad824c8195c773a438, State: Initialized, Role: FOLLOWER
I20260620 18:48:31.064110 3995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb47cfb7f6545ad824c8195c773a438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37925 } }
I20260620 18:48:31.064208 3995 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:31.064242 3995 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:31.064337 3995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:31.064515 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:31.058761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47139 (local address 127.0.0.1:37925)
0620 18:48:31.058958 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:31.058968 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:31.059003 (+ 35us) server_negotiation.cc:407] Connection header received
0620 18:48:31.059616 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:31.059626 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:31.059688 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:31.059840 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:31.060075 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:31.061449 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:31.062284 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:31.062924 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:31.063053 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:31.063794 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:31.063798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:31.063799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:31.064006 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:31.064049 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:31.064065 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:31.064144 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:31.064205 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 18:48:31.064276 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260620 18:48:31.064960 3995 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb47cfb7f6545ad824c8195c773a438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37925 } }
I20260620 18:48:31.065076 3995 leader_election.cc:302] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6eb47cfb7f6545ad824c8195c773a438; no voters:
I20260620 18:48:31.065251 3995 leader_election.cc:288] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:31.065279 4000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:31.065454 4000 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [term 1 LEADER]: Becoming Leader. State: Replica: 6eb47cfb7f6545ad824c8195c773a438, State: Running, Role: LEADER
I20260620 18:48:31.065485 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:31.065500 3995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:31.065620 4000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb47cfb7f6545ad824c8195c773a438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37925 } }
I20260620 18:48:31.065974 4000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6eb47cfb7f6545ad824c8195c773a438. Latest consensus state: current_term: 1 leader_uuid: "6eb47cfb7f6545ad824c8195c773a438" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb47cfb7f6545ad824c8195c773a438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37925 } } }
I20260620 18:48:31.066087 4000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:31.066233 4001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6eb47cfb7f6545ad824c8195c773a438" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb47cfb7f6545ad824c8195c773a438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37925 } } }
I20260620 18:48:31.066291 4001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:31.066404 4005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:31.066893 4005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:31.068634 4005 catalog_manager.cc:1347] Generated new cluster ID: 5449392f1a344ed6b7b3a538cba3ba6e
I20260620 18:48:31.068706 4005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:31.071163 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:31.067658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51763 (local address 127.0.0.1:37925)
0620 18:48:31.067693 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:31.067694 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:31.067833 (+ 139us) server_negotiation.cc:407] Connection header received
0620 18:48:31.067859 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:31.067860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:31.067871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:31.067938 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:31.068174 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:31.068852 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:31.069847 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:31.070156 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:31.070231 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:31.070890 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:31.070891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:31.070892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:31.070896 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:31.070911 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:31.070915 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:31.070974 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:31.071033 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 18:48:31.071056 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 18:48:31.077590 4005 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:31.078338 4005 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:31.085083 4005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6eb47cfb7f6545ad824c8195c773a438: Generated new TSK 0
I20260620 18:48:31.085306 4005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:31.093770 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 3927
2026-06-20T18:48:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:31 dist-test-slave-tvl9 krb5kdc[4026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:31 dist-test-slave-tvl9 krb5kdc[4026](info): set up 2 sockets
Jun 20 18:48:31 dist-test-slave-tvl9 krb5kdc[4026](info): commencing operation
krb5kdc: starting...
W20260620 18:48:32.135304 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:32 dist-test-slave-tvl9 krb5kdc[4026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:32.253980 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:32.335512 4039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:32.335769 4039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:32.335810 4039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:32.337075 4039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:32.337157 4039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:32.337180 4039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:32.337200 4039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:32.337229 4039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:32.338718 4039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:32.339015 4039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:32.339255 4039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:32.344146 4044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:32.344158 4047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:32.344313 4039 server_base.cc:1034] running on GCE node
W20260620 18:48:32.344496 4045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:32.344755 4039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:32.345010 4039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:32.346160 4039 hybrid_clock.cc:648] HybridClock initialized: now 1781981312346149 us; error 29 us; skew 500 ppm
Jun 20 18:48:32 dist-test-slave-tvl9 krb5kdc[4026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981312, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:32.353351 4039 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:32.353799 4039 webserver.cc:460] Webserver started at http://127.0.0.1:38335/ using document root <none> and password file <none>
I20260620 18:48:32.354012 4039 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:32.354056 4039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:32.354149 4039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:32.355227 4039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "b248e62ec92d4d549043600253dd4bae"
format_stamp: "Formatted at 2026-06-20 18:48:32 on dist-test-slave-tvl9"
server_key: "4f60721911a3790a6885429d0d0a6afd"
server_key_iv: "962ead8831a0b5ec6eb4adbb0fe1fc34"
server_key_version: "encryptionkey@0"
I20260620 18:48:32.355621 4039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "b248e62ec92d4d549043600253dd4bae"
format_stamp: "Formatted at 2026-06-20 18:48:32 on dist-test-slave-tvl9"
server_key: "4f60721911a3790a6885429d0d0a6afd"
server_key_iv: "962ead8831a0b5ec6eb4adbb0fe1fc34"
server_key_version: "encryptionkey@0"
I20260620 18:48:32.356935 4039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:32.357677 4054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:32.357853 4039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:32.357945 4039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "b248e62ec92d4d549043600253dd4bae"
format_stamp: "Formatted at 2026-06-20 18:48:32 on dist-test-slave-tvl9"
server_key: "4f60721911a3790a6885429d0d0a6afd"
server_key_iv: "962ead8831a0b5ec6eb4adbb0fe1fc34"
server_key_version: "encryptionkey@0"
I20260620 18:48:32.358004 4039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:32.361449 4039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:32.363451 4039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:32.363576 4039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:32.367465 4039 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36937
I20260620 18:48:32.367547 4106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36937 every 8 connection(s)
I20260620 18:48:32.367871 4039 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:32.368690 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 4039
I20260620 18:48:32.368831 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:32.369037 4107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:32.369066 15669 external_mini_cluster.cc:1428] Setting key 654a58333b89532042af68b7272040d7
I20260620 18:48:32.371831 4107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae: Bootstrap starting.
I20260620 18:48:32.372650 4107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:32.373040 4107 log.cc:824] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:32.373267 4110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:32.369638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40585 (local address 127.0.0.1:36937)
0620 18:48:32.371459 (+ 1821us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:32.371467 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:32.371493 (+ 26us) server_negotiation.cc:407] Connection header received
0620 18:48:32.371916 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:32.371924 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:32.371960 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:32.372101 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:32.372791 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:32.372795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:32.372796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:32.372801 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:32.372851 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:32.372862 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:32.372930 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:32.373017 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 18:48:32.373078 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1766,"thread_start_us":66,"threads_started":1}
I20260620 18:48:32.373869 4107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae: No bootstrap required, opened a new log
I20260620 18:48:32.374382 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:32.376068 4107 raft_consensus.cc:348] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b248e62ec92d4d549043600253dd4bae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260620 18:48:32.376228 4107 raft_consensus.cc:374] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:32.376291 4107 raft_consensus.cc:729] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b248e62ec92d4d549043600253dd4bae, State: Initialized, Role: FOLLOWER
I20260620 18:48:32.376446 4107 consensus_queue.cc:260] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b248e62ec92d4d549043600253dd4bae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260620 18:48:32.376550 4107 raft_consensus.cc:388] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:32.376621 4107 raft_consensus.cc:482] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:32.376679 4107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:32.377055 4110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:32.375864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57417 (local address 127.0.0.1:36937)
0620 18:48:32.375903 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:32.375905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:32.376038 (+ 133us) server_negotiation.cc:407] Connection header received
0620 18:48:32.376119 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:32.376120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:32.376125 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:32.376187 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:32.376860 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:32.376862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:32.376862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:32.376871 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:32.376886 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:32.376889 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:32.376935 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:32.376999 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 18:48:32.377011 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:48:32.377439 4107 raft_consensus.cc:504] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b248e62ec92d4d549043600253dd4bae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260620 18:48:32.377574 4107 leader_election.cc:302] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b248e62ec92d4d549043600253dd4bae; no voters:
I20260620 18:48:32.377736 4107 leader_election.cc:288] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:32.377775 4117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:32.377931 4117 raft_consensus.cc:686] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [term 1 LEADER]: Becoming Leader. State: Replica: b248e62ec92d4d549043600253dd4bae, State: Running, Role: LEADER
I20260620 18:48:32.378008 4107 sys_catalog.cc:564] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:32.378033 4117 consensus_queue.cc:237] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b248e62ec92d4d549043600253dd4bae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260620 18:48:32.378381 4117 sys_catalog.cc:455] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [sys.catalog]: SysCatalogTable state changed. Reason: New leader b248e62ec92d4d549043600253dd4bae. Latest consensus state: current_term: 1 leader_uuid: "b248e62ec92d4d549043600253dd4bae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b248e62ec92d4d549043600253dd4bae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } } }
I20260620 18:48:32.378463 4117 sys_catalog.cc:458] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:32.378417 4119 sys_catalog.cc:455] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b248e62ec92d4d549043600253dd4bae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b248e62ec92d4d549043600253dd4bae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } } }
I20260620 18:48:32.378635 4119 sys_catalog.cc:458] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:32.378758 4124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:32.379320 4124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:32.380925 4124 catalog_manager.cc:1347] Generated new cluster ID: 719f9207e51d4191b2335c8a38f76b1a
I20260620 18:48:32.381001 4124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:32.396041 4124 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:32.396251 4124 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:32.402964 4124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b248e62ec92d4d549043600253dd4bae: Generated new TSK 0
I20260620 18:48:32.403192 4124 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:32.407732 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 4039
2026-06-20T18:48:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:32 dist-test-slave-tvl9 krb5kdc[4138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:32 dist-test-slave-tvl9 krb5kdc[4138](info): set up 2 sockets
Jun 20 18:48:32 dist-test-slave-tvl9 krb5kdc[4138](info): commencing operation
krb5kdc: starting...
W20260620 18:48:33.447467 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:33 dist-test-slave-tvl9 krb5kdc[4138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:33.566764 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43301
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:33.648906 4151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:33.649174 4151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:33.649214 4151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:33.650389 4151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:33.650465 4151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:33.650490 4151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:33.650519 4151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:33.650544 4151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:33.652040 4151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:33.652333 4151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:33.652567 4151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:33.657475 4157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:33.657516 4159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:33.657546 4151 server_base.cc:1034] running on GCE node
W20260620 18:48:33.657552 4156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:33.657908 4151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:33.658164 4151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:33.659346 4151 hybrid_clock.cc:648] HybridClock initialized: now 1781981313659304 us; error 36 us; skew 500 ppm
Jun 20 18:48:33 dist-test-slave-tvl9 krb5kdc[4138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981313, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:33.661907 4151 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:33.662396 4151 webserver.cc:460] Webserver started at http://127.0.0.1:38829/ using document root <none> and password file <none>
I20260620 18:48:33.662622 4151 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:33.662689 4151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:33.662840 4151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:33.663990 4151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "2ad2219119064ba19a01a7986170cf52"
format_stamp: "Formatted at 2026-06-20 18:48:33 on dist-test-slave-tvl9"
server_key: "b667cdf34918f73f276007c66547b573"
server_key_iv: "a13eaac8c88aa1f259587404b6c3243b"
server_key_version: "encryptionkey@0"
I20260620 18:48:33.664361 4151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "2ad2219119064ba19a01a7986170cf52"
format_stamp: "Formatted at 2026-06-20 18:48:33 on dist-test-slave-tvl9"
server_key: "b667cdf34918f73f276007c66547b573"
server_key_iv: "a13eaac8c88aa1f259587404b6c3243b"
server_key_version: "encryptionkey@0"
I20260620 18:48:33.665709 4151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:48:33.666429 4166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:33.666613 4151 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:33.666723 4151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "2ad2219119064ba19a01a7986170cf52"
format_stamp: "Formatted at 2026-06-20 18:48:33 on dist-test-slave-tvl9"
server_key: "b667cdf34918f73f276007c66547b573"
server_key_iv: "a13eaac8c88aa1f259587404b6c3243b"
server_key_version: "encryptionkey@0"
I20260620 18:48:33.666797 4151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:33.676311 4151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:33.678164 4151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:33.678277 4151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:33.682214 4151 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43301
I20260620 18:48:33.682279 4218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43301 every 8 connection(s)
I20260620 18:48:33.682633 4151 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:33.683575 4219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:33.685827 4219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52: Bootstrap starting.
I20260620 18:48:33.686419 4219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:33.686715 4219 log.cc:824] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:33.687475 4219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52: No bootstrap required, opened a new log
I20260620 18:48:33.689078 4219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad2219119064ba19a01a7986170cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } }
I20260620 18:48:33.689211 4219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:33.689234 4219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad2219119064ba19a01a7986170cf52, State: Initialized, Role: FOLLOWER
I20260620 18:48:33.689415 4219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad2219119064ba19a01a7986170cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } }
I20260620 18:48:33.689477 4219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:33.689522 4219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:33.689571 4219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:33.690089 4219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad2219119064ba19a01a7986170cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } }
I20260620 18:48:33.690199 4219 leader_election.cc:302] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ad2219119064ba19a01a7986170cf52; no voters:
I20260620 18:48:33.690382 4219 leader_election.cc:288] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:33.690424 4222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:33.690604 4222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [term 1 LEADER]: Becoming Leader. State: Replica: 2ad2219119064ba19a01a7986170cf52, State: Running, Role: LEADER
I20260620 18:48:33.690644 4219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:33.690721 4222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad2219119064ba19a01a7986170cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } }
I20260620 18:48:33.691119 4224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ad2219119064ba19a01a7986170cf52. Latest consensus state: current_term: 1 leader_uuid: "2ad2219119064ba19a01a7986170cf52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad2219119064ba19a01a7986170cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } } }
I20260620 18:48:33.691269 4224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:33.691267 4223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ad2219119064ba19a01a7986170cf52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad2219119064ba19a01a7986170cf52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } } }
I20260620 18:48:33.691486 4223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:33.691742 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 4151
I20260620 18:48:33.691860 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:33.691871 4231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:33.692071 15669 external_mini_cluster.cc:1428] Setting key 9c4de7d96332dd150d4a2dec4f6d9f59
I20260620 18:48:33.692363 4231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:33.694538 4231 catalog_manager.cc:1347] Generated new cluster ID: 2628a4f72ae84d1db1e4877c4a5dda2f
I20260620 18:48:33.694605 4231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:48:33 dist-test-slave-tvl9 krb5kdc[4138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 18:48:33.704319 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:33.692573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:60633 (local address 127.0.0.1:43301)
0620 18:48:33.692785 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:33.692815 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:33.693020 (+ 205us) server_negotiation.cc:407] Connection header received
0620 18:48:33.693172 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:33.693180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:33.693236 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:33.693385 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:33.694149 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:33.695484 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:33.696595 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:33.697214 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:33.697315 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:33.700343 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:33.700349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:33.700349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:33.700363 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:33.702463 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:33.702911 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:33.702915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:33.702915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:33.703065 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:33.703305 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:33.703306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:33.703306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:33.703682 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:33.703848 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:33.703965 (+ 117us) server_negotiation.cc:299] Negotiation successful
0620 18:48:33.704081 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
I20260620 18:48:33.705403 4231 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:33.705648 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:33.706033 4231 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:33.713939 4231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ad2219119064ba19a01a7986170cf52: Generated new TSK 0
I20260620 18:48:33.714138 4231 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:33.714767 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:33.707161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:37313 (local address 127.0.0.1:43301)
0620 18:48:33.707194 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:33.707196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:33.707347 (+ 151us) server_negotiation.cc:407] Connection header received
0620 18:48:33.707408 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:33.707409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:33.707420 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:33.707467 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:33.708084 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:33.708850 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:33.709896 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:33.710214 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:33.710271 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:33.711753 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:33.711755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:33.711755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:33.711762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:33.713331 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:33.713777 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:33.713778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:33.713779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:33.713839 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:33.714237 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:33.714238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:33.714239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:33.714346 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:33.714465 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:33.714605 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 18:48:33.714671 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 18:48:33.718456 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 4151
W20260620 18:48:33.722916 4136 connection.cc:535] client connection to 127.0.0.1:43301 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43301): Connection reset by peer (error 104)
2026-06-20T18:48:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:33 dist-test-slave-tvl9 krb5kdc[4250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:33 dist-test-slave-tvl9 krb5kdc[4250](info): set up 2 sockets
Jun 20 18:48:33 dist-test-slave-tvl9 krb5kdc[4250](info): commencing operation
krb5kdc: starting...
W20260620 18:48:34.751623 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:34 dist-test-slave-tvl9 krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:34.872575 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:34.955308 4263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:34.955631 4263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:34.955677 4263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:34.956914 4263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:34.956995 4263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:34.957021 4263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:34.957043 4263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:34.957072 4263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:34.958609 4263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:34.958915 4263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:34.959154 4263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:34.964116 4269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:34.964119 4268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:34.964118 4271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:34.964438 4263 server_base.cc:1034] running on GCE node
I20260620 18:48:34.964674 4263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:34.964923 4263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:34.966075 4263 hybrid_clock.cc:648] HybridClock initialized: now 1781981314966065 us; error 34 us; skew 500 ppm
Jun 20 18:48:34 dist-test-slave-tvl9 krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981314, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:34.968577 4263 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:34.969067 4263 webserver.cc:460] Webserver started at http://127.0.0.1:40585/ using document root <none> and password file <none>
I20260620 18:48:34.969309 4263 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:34.969354 4263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:34.969523 4263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:34.970788 4263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "1c3a9dff35b640d995012bd55b0fb2a0"
format_stamp: "Formatted at 2026-06-20 18:48:34 on dist-test-slave-tvl9"
server_key: "d5895b6b8dd57304928c113122ddf34e"
server_key_iv: "59c5f9c58577ae4a272baf8c283bf700"
server_key_version: "encryptionkey@0"
I20260620 18:48:34.971251 4263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "1c3a9dff35b640d995012bd55b0fb2a0"
format_stamp: "Formatted at 2026-06-20 18:48:34 on dist-test-slave-tvl9"
server_key: "d5895b6b8dd57304928c113122ddf34e"
server_key_iv: "59c5f9c58577ae4a272baf8c283bf700"
server_key_version: "encryptionkey@0"
I20260620 18:48:34.972666 4263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:34.973438 4278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:34.973618 4263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:34.973722 4263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "1c3a9dff35b640d995012bd55b0fb2a0"
format_stamp: "Formatted at 2026-06-20 18:48:34 on dist-test-slave-tvl9"
server_key: "d5895b6b8dd57304928c113122ddf34e"
server_key_iv: "59c5f9c58577ae4a272baf8c283bf700"
server_key_version: "encryptionkey@0"
I20260620 18:48:34.973807 4263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:34.988683 4263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:34.990633 4263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:34.990805 4263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:34.995086 4263 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36853
I20260620 18:48:34.995123 4330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36853 every 8 connection(s)
I20260620 18:48:34.995502 4263 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:34.996410 4331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:34.997714 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 4263
I20260620 18:48:34.997839 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:34.998011 15669 external_mini_cluster.cc:1428] Setting key ffa37141a7ff592eb8a63b1b08f7d964
I20260620 18:48:34.999143 4331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0: Bootstrap starting.
I20260620 18:48:34.999931 4331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:35.000238 4331 log.cc:824] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:35.001024 4331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0: No bootstrap required, opened a new log
I20260620 18:48:35.003386 4331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3a9dff35b640d995012bd55b0fb2a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } }
I20260620 18:48:35.003582 4331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:35.003625 4331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c3a9dff35b640d995012bd55b0fb2a0, State: Initialized, Role: FOLLOWER
I20260620 18:48:35.003816 4331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3a9dff35b640d995012bd55b0fb2a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } }
I20260620 18:48:35.003890 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:34.998590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:58179 (local address 127.0.0.1:36853)
0620 18:48:34.998768 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:34.998777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:34.998803 (+ 26us) server_negotiation.cc:407] Connection header received
0620 18:48:34.999225 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:34.999231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:34.999376 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:34.999499 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:34.999778 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:35.000745 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:35.001853 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:35.002276 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:35.002355 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:35.003144 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:35.003148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:35.003148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:35.003310 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:35.003400 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:35.003413 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:35.003480 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:35.003649 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 18:48:35.003683 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260620 18:48:35.003925 4331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:35.003968 4331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:35.004024 4331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:35.004596 4331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3a9dff35b640d995012bd55b0fb2a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } }
I20260620 18:48:35.004701 4331 leader_election.cc:302] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c3a9dff35b640d995012bd55b0fb2a0; no voters:
I20260620 18:48:35.004863 4331 leader_election.cc:288] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:35.004882 4336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:35.004992 4336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [term 1 LEADER]: Becoming Leader. State: Replica: 1c3a9dff35b640d995012bd55b0fb2a0, State: Running, Role: LEADER
I20260620 18:48:35.005096 4336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3a9dff35b640d995012bd55b0fb2a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } }
I20260620 18:48:35.005117 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:35.005129 4331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:35.005460 4337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c3a9dff35b640d995012bd55b0fb2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3a9dff35b640d995012bd55b0fb2a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } } }
I20260620 18:48:35.005551 4337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:35.005724 4338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c3a9dff35b640d995012bd55b0fb2a0. Latest consensus state: current_term: 1 leader_uuid: "1c3a9dff35b640d995012bd55b0fb2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3a9dff35b640d995012bd55b0fb2a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36853 } } }
I20260620 18:48:35.005795 4338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0 [sys.catalog]: This master's current role is: LEADER
W20260620 18:48:35.007113 4356 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:35.007174 4356 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:35.007223 4341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:35.007428 4341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:35.008813 4341 catalog_manager.cc:1347] Generated new cluster ID: 78fc383762b648918df97356d588eb21
I20260620 18:48:35.008862 4341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:35.010516 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:35.006881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:50417 (local address 127.0.0.1:36853)
0620 18:48:35.006920 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:35.006922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:35.006931 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:35.007012 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:35.007013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:35.007026 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:35.007059 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:35.007282 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:35.008064 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:35.009061 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:35.009380 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:35.009422 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:35.010229 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:35.010230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:35.010230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:35.010235 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:35.010249 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:35.010253 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:35.010286 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:35.010420 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 18:48:35.010431 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:48:35.023386 4341 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:35.024149 4341 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:35.035696 4341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c3a9dff35b640d995012bd55b0fb2a0: Generated new TSK 0
I20260620 18:48:35.035928 4341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:35.039664 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 4263
W20260620 18:48:35.044174 4248 connection.cc:535] client connection to 127.0.0.1:36853 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36853): Connection reset by peer (error 104)
2026-06-20T18:48:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:35 dist-test-slave-tvl9 krb5kdc[4362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:35 dist-test-slave-tvl9 krb5kdc[4362](info): set up 2 sockets
Jun 20 18:48:35 dist-test-slave-tvl9 krb5kdc[4362](info): commencing operation
krb5kdc: starting...
W20260620 18:48:36.071566 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:36 dist-test-slave-tvl9 krb5kdc[4362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:36.192178 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:36.273824 4375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:36.274097 4375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:36.274137 4375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:36.275350 4375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:36.275429 4375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:36.275452 4375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:36.275473 4375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:36.275504 4375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:36.277037 4375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:36.277323 4375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:36.277551 4375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:36.282450 4380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:36.282495 4381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:36.282459 4383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:36.282665 4375 server_base.cc:1034] running on GCE node
I20260620 18:48:36.282910 4375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:36.283213 4375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:36.284382 4375 hybrid_clock.cc:648] HybridClock initialized: now 1781981316284381 us; error 39 us; skew 500 ppm
Jun 20 18:48:36 dist-test-slave-tvl9 krb5kdc[4362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981316, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:36.286916 4375 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:36.287433 4375 webserver.cc:460] Webserver started at http://127.0.0.1:38377/ using document root <none> and password file <none>
I20260620 18:48:36.287683 4375 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:36.287751 4375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:36.287885 4375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:36.288969 4375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "b713de60ef264ef8a9b1c589458bb41f"
format_stamp: "Formatted at 2026-06-20 18:48:36 on dist-test-slave-tvl9"
server_key: "d2eede664fb57b3c3f32fdbe58854883"
server_key_iv: "6131f55a4433dfa6b438ed9ca0d080f4"
server_key_version: "encryptionkey@0"
I20260620 18:48:36.289338 4375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "b713de60ef264ef8a9b1c589458bb41f"
format_stamp: "Formatted at 2026-06-20 18:48:36 on dist-test-slave-tvl9"
server_key: "d2eede664fb57b3c3f32fdbe58854883"
server_key_iv: "6131f55a4433dfa6b438ed9ca0d080f4"
server_key_version: "encryptionkey@0"
I20260620 18:48:36.290709 4375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 18:48:36.291539 4390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:36.291718 4375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:36.291821 4375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "b713de60ef264ef8a9b1c589458bb41f"
format_stamp: "Formatted at 2026-06-20 18:48:36 on dist-test-slave-tvl9"
server_key: "d2eede664fb57b3c3f32fdbe58854883"
server_key_iv: "6131f55a4433dfa6b438ed9ca0d080f4"
server_key_version: "encryptionkey@0"
I20260620 18:48:36.291910 4375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:36.295118 4375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:36.297180 4375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:36.297348 4375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:36.301563 4375 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36941
I20260620 18:48:36.301611 4442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36941 every 8 connection(s)
I20260620 18:48:36.301996 4375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:36.302923 4443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:36.305330 4443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f: Bootstrap starting.
I20260620 18:48:36.305938 4443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:36.306221 4443 log.cc:824] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:36.306866 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 4375
I20260620 18:48:36.306985 4443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f: No bootstrap required, opened a new log
I20260620 18:48:36.307008 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:36.307178 15669 external_mini_cluster.cc:1428] Setting key f8c4f44c659f51161518d79472af62a9
I20260620 18:48:36.309391 4443 raft_consensus.cc:348] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713de60ef264ef8a9b1c589458bb41f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36941 } }
I20260620 18:48:36.309628 4443 raft_consensus.cc:374] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:36.309693 4443 raft_consensus.cc:729] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b713de60ef264ef8a9b1c589458bb41f, State: Initialized, Role: FOLLOWER
I20260620 18:48:36.309823 4443 consensus_queue.cc:260] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713de60ef264ef8a9b1c589458bb41f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36941 } }
I20260620 18:48:36.309926 4443 raft_consensus.cc:388] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:36.309959 4443 raft_consensus.cc:482] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:36.310015 4443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:36.310024 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:36.307797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:51281 (local address 127.0.0.1:36941)
0620 18:48:36.308008 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:36.308019 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:36.308053 (+ 34us) server_negotiation.cc:407] Connection header received
0620 18:48:36.308485 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:36.308494 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:36.308528 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:36.308683 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:36.309339 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:36.309343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:36.309343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:36.309348 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:36.309391 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:36.309406 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:36.309481 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:36.309534 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 18:48:36.309577 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260620 18:48:36.310793 4443 raft_consensus.cc:504] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713de60ef264ef8a9b1c589458bb41f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36941 } }
I20260620 18:48:36.310925 4443 leader_election.cc:302] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b713de60ef264ef8a9b1c589458bb41f; no voters:
I20260620 18:48:36.310982 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:36.311088 4443 leader_election.cc:288] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:36.311121 4448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:36.311276 4448 raft_consensus.cc:686] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [term 1 LEADER]: Becoming Leader. State: Replica: b713de60ef264ef8a9b1c589458bb41f, State: Running, Role: LEADER
I20260620 18:48:36.311347 4443 sys_catalog.cc:564] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:36.311422 4448 consensus_queue.cc:237] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713de60ef264ef8a9b1c589458bb41f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36941 } }
I20260620 18:48:36.311750 4448 sys_catalog.cc:455] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b713de60ef264ef8a9b1c589458bb41f. Latest consensus state: current_term: 1 leader_uuid: "b713de60ef264ef8a9b1c589458bb41f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713de60ef264ef8a9b1c589458bb41f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36941 } } }
I20260620 18:48:36.311846 4448 sys_catalog.cc:458] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:36.311762 4450 sys_catalog.cc:455] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b713de60ef264ef8a9b1c589458bb41f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713de60ef264ef8a9b1c589458bb41f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36941 } } }
I20260620 18:48:36.311946 4450 sys_catalog.cc:458] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:36.312178 4455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:36.312738 4455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:36.316053 4455 catalog_manager.cc:1347] Generated new cluster ID: ed4255df241d4aa8ade3fe202ab662ad
I20260620 18:48:36.316107 4455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:36.316752 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:36.312632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:52883 (local address 127.0.0.1:36941)
0620 18:48:36.312668 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:36.312670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:36.315672 (+ 3002us) server_negotiation.cc:407] Connection header received
0620 18:48:36.315739 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:36.315740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:36.315746 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:36.315818 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:36.316535 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:36.316536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:36.316537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:36.316542 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:36.316558 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:36.316562 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:36.316624 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:36.316679 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 18:48:36.316692 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 18:48:36.324765 4455 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:36.324934 4455 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:36.332211 4455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b713de60ef264ef8a9b1c589458bb41f: Generated new TSK 0
I20260620 18:48:36.332410 4455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:36.337666 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 4375
2026-06-20T18:48:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:36 dist-test-slave-tvl9 krb5kdc[4474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:36 dist-test-slave-tvl9 krb5kdc[4474](info): set up 2 sockets
Jun 20 18:48:36 dist-test-slave-tvl9 krb5kdc[4474](info): commencing operation
krb5kdc: starting...
W20260620 18:48:37.371572 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:37 dist-test-slave-tvl9 krb5kdc[4474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:37.489686 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:37.570096 4487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:37.570298 4487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:37.570320 4487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:37.571410 4487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:37.571444 4487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:37.571458 4487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:37.571471 4487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:37.571483 4487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:37.572886 4487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37751
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:37.573102 4487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:37.573271 4487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:48:37.578132 4487 server_base.cc:1034] running on GCE node
W20260620 18:48:37.578097 4492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:37.578080 4495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:37.578083 4493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:37.578670 4487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:37.579010 4487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:37.580163 4487 hybrid_clock.cc:648] HybridClock initialized: now 1781981317580142 us; error 39 us; skew 500 ppm
Jun 20 18:48:37 dist-test-slave-tvl9 krb5kdc[4474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981317, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:37.586300 4487 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:37.586697 4487 webserver.cc:460] Webserver started at http://127.0.0.1:34163/ using document root <none> and password file <none>
I20260620 18:48:37.586898 4487 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:37.586936 4487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:37.587033 4487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:37.588115 4487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "9435fefc91d3468397a4f6306b0b8bf0"
format_stamp: "Formatted at 2026-06-20 18:48:37 on dist-test-slave-tvl9"
server_key: "e744e34373f1c2f663254f5dd28af0ba"
server_key_iv: "6c40820e99ec4d23ed8964f89345fc18"
server_key_version: "encryptionkey@0"
I20260620 18:48:37.588402 4487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "9435fefc91d3468397a4f6306b0b8bf0"
format_stamp: "Formatted at 2026-06-20 18:48:37 on dist-test-slave-tvl9"
server_key: "e744e34373f1c2f663254f5dd28af0ba"
server_key_iv: "6c40820e99ec4d23ed8964f89345fc18"
server_key_version: "encryptionkey@0"
I20260620 18:48:37.589646 4487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 18:48:37.590329 4502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:37.590502 4487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:48:37.590610 4487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "9435fefc91d3468397a4f6306b0b8bf0"
format_stamp: "Formatted at 2026-06-20 18:48:37 on dist-test-slave-tvl9"
server_key: "e744e34373f1c2f663254f5dd28af0ba"
server_key_iv: "6c40820e99ec4d23ed8964f89345fc18"
server_key_version: "encryptionkey@0"
I20260620 18:48:37.590687 4487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:37.604367 4487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:37.606411 4487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:37.606595 4487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:37.610845 4487 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37751
I20260620 18:48:37.610881 4554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37751 every 8 connection(s)
I20260620 18:48:37.611234 4487 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:37.612126 4555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:37.614323 4555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0: Bootstrap starting.
I20260620 18:48:37.614593 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 4487
I20260620 18:48:37.614720 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:37.614910 15669 external_mini_cluster.cc:1428] Setting key cd6ec96959dbe8dc490f6577f8a0da90
I20260620 18:48:37.615078 4555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:37.615442 4555 log.cc:824] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:37.616122 4555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0: No bootstrap required, opened a new log
I20260620 18:48:37.617913 4555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9435fefc91d3468397a4f6306b0b8bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } }
I20260620 18:48:37.618075 4555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:37.618124 4555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9435fefc91d3468397a4f6306b0b8bf0, State: Initialized, Role: FOLLOWER
I20260620 18:48:37.618255 4555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9435fefc91d3468397a4f6306b0b8bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } }
I20260620 18:48:37.618317 4555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:37.618369 4555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:37.618419 4555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:37.618932 4555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9435fefc91d3468397a4f6306b0b8bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } }
I20260620 18:48:37.619055 4555 leader_election.cc:302] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9435fefc91d3468397a4f6306b0b8bf0; no voters:
I20260620 18:48:37.619223 4555 leader_election.cc:288] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:37.619256 4560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:37.619457 4560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [term 1 LEADER]: Becoming Leader. State: Replica: 9435fefc91d3468397a4f6306b0b8bf0, State: Running, Role: LEADER
I20260620 18:48:37.619539 4555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:37.619649 4560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9435fefc91d3468397a4f6306b0b8bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } }
I20260620 18:48:37.620047 4560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9435fefc91d3468397a4f6306b0b8bf0. Latest consensus state: current_term: 1 leader_uuid: "9435fefc91d3468397a4f6306b0b8bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9435fefc91d3468397a4f6306b0b8bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } } }
I20260620 18:48:37.620055 4562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9435fefc91d3468397a4f6306b0b8bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9435fefc91d3468397a4f6306b0b8bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37751 } } }
I20260620 18:48:37.620138 4560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:37.620153 4562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:37.620469 4564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:37.621193 4564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 18:48:37 dist-test-slave-tvl9 krb5kdc[4474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 18:48:37.624130 4564 catalog_manager.cc:1347] Generated new cluster ID: 7f0c0ad7e7aa458aaab07856a8a2ee82
I20260620 18:48:37.624181 4564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:37.626241 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:37.615433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:35681 (local address 127.0.0.1:37751)
0620 18:48:37.615657 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:37.615668 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:37.615701 (+ 33us) server_negotiation.cc:407] Connection header received
0620 18:48:37.616186 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:37.616193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:37.616279 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:37.616433 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:37.617088 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:37.617973 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:37.618876 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:37.619518 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:37.619938 (+ 420us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:37.622476 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:37.622481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:37.622481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:37.622506 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:37.624393 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:37.624814 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:37.624817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:37.624817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:37.624909 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:37.625108 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:37.625108 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:37.625109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:37.625373 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:37.625890 (+ 517us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:37.625968 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 18:48:37.626058 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":86,"threads_started":1}
I20260620 18:48:37.627475 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:37.638788 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:37.631596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:58321 (local address 127.0.0.1:37751)
0620 18:48:37.631636 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:37.631638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:37.631799 (+ 161us) server_negotiation.cc:407] Connection header received
0620 18:48:37.631856 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:37.631857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:37.631866 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:37.631920 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:37.632518 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:37.633244 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:37.634263 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:37.634569 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:37.634625 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:37.636040 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:37.636042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:37.636042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:37.636049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:37.637556 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:37.637986 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:37.637987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:37.637987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:37.638044 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:37.638273 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:37.638274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:37.638274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:37.638389 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:37.638480 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:37.638636 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 18:48:37.638696 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:48:37.645866 4564 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:37.646374 4564 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:37.653179 4564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9435fefc91d3468397a4f6306b0b8bf0: Generated new TSK 0
I20260620 18:48:37.653394 4564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:37.659241 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 4487
W20260620 18:48:37.663821 4472 connection.cc:535] client connection to 127.0.0.1:37751 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37751): Connection reset by peer (error 104)
2026-06-20T18:48:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:37 dist-test-slave-tvl9 krb5kdc[4586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:37 dist-test-slave-tvl9 krb5kdc[4586](info): set up 2 sockets
Jun 20 18:48:37 dist-test-slave-tvl9 krb5kdc[4586](info): commencing operation
krb5kdc: starting...
W20260620 18:48:38.691497 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:38 dist-test-slave-tvl9 krb5kdc[4586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:38.818184 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:38.901399 4599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:38.901633 4599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:38.901659 4599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:38.902829 4599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:38.902868 4599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:38.902882 4599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:38.902895 4599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:38.902907 4599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:38.904439 4599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35889
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:38.904664 4599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:38.904858 4599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:38.909833 4605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:38.909904 4607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:38.909833 4604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:38.910080 4599 server_base.cc:1034] running on GCE node
I20260620 18:48:38.910356 4599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:38.910663 4599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:38.911820 4599 hybrid_clock.cc:648] HybridClock initialized: now 1781981318911792 us; error 47 us; skew 500 ppm
Jun 20 18:48:38 dist-test-slave-tvl9 krb5kdc[4586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981318, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:38.914306 4599 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:38.914832 4599 webserver.cc:460] Webserver started at http://127.0.0.1:46589/ using document root <none> and password file <none>
I20260620 18:48:38.915087 4599 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:38.915159 4599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:38.915360 4599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:38.916469 4599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "1890a520ac444363b2f115ac1c693812"
format_stamp: "Formatted at 2026-06-20 18:48:38 on dist-test-slave-tvl9"
server_key: "06f6201280bdc02e354a20bdb86fd2cf"
server_key_iv: "f0a7f02fac6950bc4a69b9be47a5233a"
server_key_version: "encryptionkey@0"
I20260620 18:48:38.916839 4599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "1890a520ac444363b2f115ac1c693812"
format_stamp: "Formatted at 2026-06-20 18:48:38 on dist-test-slave-tvl9"
server_key: "06f6201280bdc02e354a20bdb86fd2cf"
server_key_iv: "f0a7f02fac6950bc4a69b9be47a5233a"
server_key_version: "encryptionkey@0"
I20260620 18:48:38.918237 4599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:38.918970 4614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:38.919145 4599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:38.919250 4599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "1890a520ac444363b2f115ac1c693812"
format_stamp: "Formatted at 2026-06-20 18:48:38 on dist-test-slave-tvl9"
server_key: "06f6201280bdc02e354a20bdb86fd2cf"
server_key_iv: "f0a7f02fac6950bc4a69b9be47a5233a"
server_key_version: "encryptionkey@0"
I20260620 18:48:38.919370 4599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:38.930176 4599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:38.932214 4599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:38.932380 4599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:38.936591 4599 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35889
I20260620 18:48:38.936628 4666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35889 every 8 connection(s)
I20260620 18:48:38.936985 4599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:38.937883 4667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:38.940311 4667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812: Bootstrap starting.
I20260620 18:48:38.940913 4667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:38.941207 4667 log.cc:824] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:38.941905 4667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812: No bootstrap required, opened a new log
I20260620 18:48:38.943394 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 4599
I20260620 18:48:38.943507 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:38.943711 15669 external_mini_cluster.cc:1428] Setting key 2cdc0a38aa97ea041f600a979245f8e5
I20260620 18:48:38.943836 4667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1890a520ac444363b2f115ac1c693812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35889 } }
I20260620 18:48:38.943990 4667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:38.944013 4667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1890a520ac444363b2f115ac1c693812, State: Initialized, Role: FOLLOWER
I20260620 18:48:38.944208 4667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1890a520ac444363b2f115ac1c693812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35889 } }
I20260620 18:48:38.944309 4667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:38.944342 4667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:38.944437 4667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:38.945633 4667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1890a520ac444363b2f115ac1c693812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35889 } }
I20260620 18:48:38.945745 4667 leader_election.cc:302] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1890a520ac444363b2f115ac1c693812; no voters:
I20260620 18:48:38.945894 4667 leader_election.cc:288] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:38.945960 4672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:38.946127 4672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [term 1 LEADER]: Becoming Leader. State: Replica: 1890a520ac444363b2f115ac1c693812, State: Running, Role: LEADER
I20260620 18:48:38.946130 4667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:38.946280 4672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1890a520ac444363b2f115ac1c693812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35889 } }
I20260620 18:48:38.946739 4673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1890a520ac444363b2f115ac1c693812" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1890a520ac444363b2f115ac1c693812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35889 } } }
I20260620 18:48:38.946943 4673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:38.946882 4674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1890a520ac444363b2f115ac1c693812. Latest consensus state: current_term: 1 leader_uuid: "1890a520ac444363b2f115ac1c693812" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1890a520ac444363b2f115ac1c693812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35889 } } }
I20260620 18:48:38.947180 4674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:38.947542 4681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:38.948086 4681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:38.949815 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:38.944208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:35691 (local address 127.0.0.1:35889)
0620 18:48:38.944434 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:38.944443 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:38.944477 (+ 34us) server_negotiation.cc:407] Connection header received
0620 18:48:38.944596 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:38.944603 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:38.944661 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:38.944817 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:38.944988 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:38.946399 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:38.947612 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:38.948234 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:38.948340 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:38.949138 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:38.949142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:38.949142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:38.949338 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:38.949376 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:38.949390 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:38.949477 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:38.949616 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 18:48:38.949646 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":88,"threads_started":1}
I20260620 18:48:38.949981 4681 catalog_manager.cc:1347] Generated new cluster ID: 6654c2091ee94f99bfa34cbe98b2766d
I20260620 18:48:38.950064 4681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:38.950698 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:38.958669 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:38.955662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:56861 (local address 127.0.0.1:35889)
0620 18:48:38.955704 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:38.955706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:38.955857 (+ 151us) server_negotiation.cc:407] Connection header received
0620 18:48:38.955931 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:38.955931 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:38.955943 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:38.955994 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:38.956133 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:38.956630 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:38.957383 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:38.957619 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:38.957703 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:38.958388 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:38.958389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:38.958390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:38.958394 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:38.958408 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:38.958413 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:38.958491 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:38.958558 (+ 67us) server_negotiation.cc:299] Negotiation successful
0620 18:48:38.958571 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:48:38.973353 4681 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:38.973922 4681 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:38.979964 4681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1890a520ac444363b2f115ac1c693812: Generated new TSK 0
I20260620 18:48:38.980185 4681 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:38.987596 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 4599
W20260620 18:48:38.992219 4584 connection.cc:535] client connection to 127.0.0.1:35889 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35889): Connection reset by peer (error 104)
2026-06-20T18:48:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:39 dist-test-slave-tvl9 krb5kdc[4698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:39 dist-test-slave-tvl9 krb5kdc[4698](info): set up 2 sockets
Jun 20 18:48:39 dist-test-slave-tvl9 krb5kdc[4698](info): commencing operation
krb5kdc: starting...
W20260620 18:48:40.023535 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:40 dist-test-slave-tvl9 krb5kdc[4698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 18:48:40.140475 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:40.222393 4711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:40.222658 4711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:40.222699 4711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:40.223892 4711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:40.223964 4711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:40.223996 4711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:40.224028 4711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:40.224054 4711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:40.225553 4711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39975
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:40.225837 4711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:40.226076 4711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:40.230867 4719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:40.230891 4716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:40.230887 4717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:40.231150 4711 server_base.cc:1034] running on GCE node
I20260620 18:48:40.231421 4711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:40.231719 4711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:40.232904 4711 hybrid_clock.cc:648] HybridClock initialized: now 1781981320232892 us; error 27 us; skew 500 ppm
Jun 20 18:48:40 dist-test-slave-tvl9 krb5kdc[4698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981320, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:40.239517 4711 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 18:48:40.239974 4711 webserver.cc:460] Webserver started at http://127.0.0.1:43193/ using document root <none> and password file <none>
I20260620 18:48:40.240234 4711 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:40.240305 4711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:40.240455 4711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:40.241515 4711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "aac0bd91a7e949f4be19387e8d80660e"
format_stamp: "Formatted at 2026-06-20 18:48:40 on dist-test-slave-tvl9"
server_key: "b947a9bc24e4991a01d50f1bee64d7c6"
server_key_iv: "bd81d6cca035c68663e0dc4c62c8e789"
server_key_version: "encryptionkey@0"
I20260620 18:48:40.241894 4711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "aac0bd91a7e949f4be19387e8d80660e"
format_stamp: "Formatted at 2026-06-20 18:48:40 on dist-test-slave-tvl9"
server_key: "b947a9bc24e4991a01d50f1bee64d7c6"
server_key_iv: "bd81d6cca035c68663e0dc4c62c8e789"
server_key_version: "encryptionkey@0"
I20260620 18:48:40.243213 4711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:40.243999 4726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:40.244172 4711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:48:40.244282 4711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "aac0bd91a7e949f4be19387e8d80660e"
format_stamp: "Formatted at 2026-06-20 18:48:40 on dist-test-slave-tvl9"
server_key: "b947a9bc24e4991a01d50f1bee64d7c6"
server_key_iv: "bd81d6cca035c68663e0dc4c62c8e789"
server_key_version: "encryptionkey@0"
I20260620 18:48:40.244357 4711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:40.247788 4711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:40.250164 4711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:40.250321 4711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:40.254182 4711 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39975
I20260620 18:48:40.254244 4778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39975 every 8 connection(s)
I20260620 18:48:40.254621 4711 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:40.255584 4779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:40.255645 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 4711
I20260620 18:48:40.255743 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:40.255931 15669 external_mini_cluster.cc:1428] Setting key 936d83960eceb3302bff2531c44efdec
I20260620 18:48:40.258314 4782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:40.256456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:57305 (local address 127.0.0.1:39975)
0620 18:48:40.256634 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:40.256641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:40.256666 (+ 25us) server_negotiation.cc:407] Connection header received
0620 18:48:40.257117 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:40.257123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:40.257148 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:40.257261 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:40.257879 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:40.257882 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:40.257882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:40.257886 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:40.257922 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:40.257931 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:40.257986 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:40.258070 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 18:48:40.258127 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260620 18:48:40.259073 4779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e: Bootstrap starting.
I20260620 18:48:40.259292 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:40.259807 4779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:40.260092 4779 log.cc:824] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:40.260841 4779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e: No bootstrap required, opened a new log
I20260620 18:48:40.261891 4782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:40.260900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:52623 (local address 127.0.0.1:39975)
0620 18:48:40.260944 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:40.260946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:40.261061 (+ 115us) server_negotiation.cc:407] Connection header received
0620 18:48:40.261132 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:40.261132 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:40.261136 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:40.261184 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:40.261667 (+ 483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:40.261668 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:40.261668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:40.261671 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:40.261683 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:40.261686 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:40.261722 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:40.261783 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 18:48:40.261793 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:48:40.263234 4779 raft_consensus.cc:348] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac0bd91a7e949f4be19387e8d80660e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } }
I20260620 18:48:40.263455 4779 raft_consensus.cc:374] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:40.263513 4779 raft_consensus.cc:729] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aac0bd91a7e949f4be19387e8d80660e, State: Initialized, Role: FOLLOWER
I20260620 18:48:40.263649 4779 consensus_queue.cc:260] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac0bd91a7e949f4be19387e8d80660e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } }
I20260620 18:48:40.263756 4779 raft_consensus.cc:388] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:40.263821 4779 raft_consensus.cc:482] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:40.263873 4779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:40.264442 4779 raft_consensus.cc:504] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac0bd91a7e949f4be19387e8d80660e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } }
I20260620 18:48:40.264565 4779 leader_election.cc:302] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aac0bd91a7e949f4be19387e8d80660e; no voters:
I20260620 18:48:40.264747 4779 leader_election.cc:288] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:40.264786 4789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:40.264940 4789 raft_consensus.cc:686] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [term 1 LEADER]: Becoming Leader. State: Replica: aac0bd91a7e949f4be19387e8d80660e, State: Running, Role: LEADER
I20260620 18:48:40.265017 4779 sys_catalog.cc:564] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:40.265060 4789 consensus_queue.cc:237] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac0bd91a7e949f4be19387e8d80660e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } }
I20260620 18:48:40.265486 4791 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [sys.catalog]: SysCatalogTable state changed. Reason: New leader aac0bd91a7e949f4be19387e8d80660e. Latest consensus state: current_term: 1 leader_uuid: "aac0bd91a7e949f4be19387e8d80660e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac0bd91a7e949f4be19387e8d80660e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } } }
I20260620 18:48:40.265589 4791 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:40.265901 4790 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aac0bd91a7e949f4be19387e8d80660e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac0bd91a7e949f4be19387e8d80660e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39975 } } }
I20260620 18:48:40.266032 4790 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e [sys.catalog]: This master's current role is: LEADER
W20260620 18:48:40.266805 4805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:40.266883 4805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:40.267014 4797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:40.267225 4797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:40.268836 4797 catalog_manager.cc:1347] Generated new cluster ID: 9beb1f56801d41db9521c6d82ef85bb4
I20260620 18:48:40.268903 4797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:40.277822 4797 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:40.278008 4797 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:40.285708 4797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aac0bd91a7e949f4be19387e8d80660e: Generated new TSK 0
I20260620 18:48:40.285908 4797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:40.292505 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 4711
2026-06-20T18:48:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:40 dist-test-slave-tvl9 krb5kdc[4810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:40 dist-test-slave-tvl9 krb5kdc[4810](info): set up 2 sockets
Jun 20 18:48:40 dist-test-slave-tvl9 krb5kdc[4810](info): commencing operation
krb5kdc: starting...
W20260620 18:48:41.343863 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:41 dist-test-slave-tvl9 krb5kdc[4810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:41.461552 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:44233
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:36133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:44233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:41.541915 4823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:41.542138 4823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:41.542162 4823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:41.543270 4823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:41.543308 4823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:41.543408 4823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:41.543437 4823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:41.543450 4823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:41.544945 4823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:36133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:44233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:44233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:41.545154 4823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:41.545331 4823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:41.550185 4831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:41.550262 4829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:41.550370 4823 server_base.cc:1034] running on GCE node
W20260620 18:48:41.550278 4828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:41.550673 4823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:41.550981 4823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:41.552115 4823 hybrid_clock.cc:648] HybridClock initialized: now 1781981321552105 us; error 27 us; skew 500 ppm
Jun 20 18:48:41 dist-test-slave-tvl9 krb5kdc[4810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:41.554507 4823 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:41.554973 4823 webserver.cc:460] Webserver started at http://127.15.77.126:39843/ using document root <none> and password file <none>
I20260620 18:48:41.555210 4823 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:41.555279 4823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:41.555451 4823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:41.556499 4823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05"
format_stamp: "Formatted at 2026-06-20 18:48:41 on dist-test-slave-tvl9"
server_key: "499fbcb5e88b5f5eeb6b503968153696"
server_key_iv: "fc7f4b84b93ec6245ba68e4842a79a1f"
server_key_version: "encryptionkey@0"
I20260620 18:48:41.556862 4823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05"
format_stamp: "Formatted at 2026-06-20 18:48:41 on dist-test-slave-tvl9"
server_key: "499fbcb5e88b5f5eeb6b503968153696"
server_key_iv: "fc7f4b84b93ec6245ba68e4842a79a1f"
server_key_version: "encryptionkey@0"
I20260620 18:48:41.558298 4823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:41.559007 4838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:41.559187 4823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:41.559293 4823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05"
format_stamp: "Formatted at 2026-06-20 18:48:41 on dist-test-slave-tvl9"
server_key: "499fbcb5e88b5f5eeb6b503968153696"
server_key_iv: "fc7f4b84b93ec6245ba68e4842a79a1f"
server_key_version: "encryptionkey@0"
I20260620 18:48:41.559424 4823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:41.569895 4823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:41.571928 4823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:41.572084 4823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:41.576115 4823 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:44233
I20260620 18:48:41.576152 4890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:44233 every 8 connection(s)
I20260620 18:48:41.576517 4823 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:41.577392 4891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:41.579716 4891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05: Bootstrap starting.
I20260620 18:48:41.580329 4891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:41.580619 4891 log.cc:824] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:41.581313 4891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05: No bootstrap required, opened a new log
I20260620 18:48:41.582944 4891 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 44233 } }
I20260620 18:48:41.583094 4891 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:41.583118 4891 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2bf68ef8cf54f5abb3e8d3ed2d27d05, State: Initialized, Role: FOLLOWER
I20260620 18:48:41.583290 4891 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 44233 } }
I20260620 18:48:41.583406 4891 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:41.583431 4891 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:41.583493 4891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:41.584355 4891 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 44233 } }
I20260620 18:48:41.584468 4891 leader_election.cc:302] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2bf68ef8cf54f5abb3e8d3ed2d27d05; no voters:
I20260620 18:48:41.584654 4891 leader_election.cc:288] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:41.584807 4894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:41.584906 4891 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:41.584978 4894 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [term 1 LEADER]: Becoming Leader. State: Replica: b2bf68ef8cf54f5abb3e8d3ed2d27d05, State: Running, Role: LEADER
I20260620 18:48:41.585073 4894 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 44233 } }
I20260620 18:48:41.585649 4895 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 44233 } } }
I20260620 18:48:41.585666 4896 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2bf68ef8cf54f5abb3e8d3ed2d27d05. Latest consensus state: current_term: 1 leader_uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2bf68ef8cf54f5abb3e8d3ed2d27d05" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 44233 } } }
I20260620 18:48:41.585764 4895 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:41.585780 4896 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:41.586002 4903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:41.586473 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 4823
I20260620 18:48:41.586601 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:41.586594 4903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:41.586779 15669 external_mini_cluster.cc:1428] Setting key 63b5969fc2a17574c1417a13423f1cbc
I20260620 18:48:41.588407 4903 catalog_manager.cc:1347] Generated new cluster ID: e7a2d0e54aba4ae9806a760282006f4f
I20260620 18:48:41.588465 4903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 18:48:41 dist-test-slave-tvl9 krb5kdc[4810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:41.596930 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:41.587458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56513 (local address 127.15.77.126:44233)
0620 18:48:41.587608 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:41.587617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:41.587650 (+ 33us) server_negotiation.cc:407] Connection header received
0620 18:48:41.587762 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:41.587768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:41.587823 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:41.587957 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:41.588716 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:41.589638 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:41.590407 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:41.590839 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:41.590981 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:41.593413 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:41.593417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:41.593418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:41.593431 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:41.595175 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:41.595616 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:41.595625 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:41.595626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:41.595726 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:41.595973 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:41.595973 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:41.595974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:41.596230 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:41.596423 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:41.596557 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 18:48:41.596759 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":85,"threads_started":1}
I20260620 18:48:41.597874 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:41.605751 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:41.599401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56957 (local address 127.15.77.126:44233)
0620 18:48:41.599454 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:41.599456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:41.599693 (+ 237us) server_negotiation.cc:407] Connection header received
0620 18:48:41.599768 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:41.599769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:41.599778 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:41.599833 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:41.600338 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:41.600838 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:41.601569 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:41.601803 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:41.601887 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:41.603104 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:41.603105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:41.603106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:41.603112 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:41.604560 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:41.604932 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:41.604933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:41.604934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:41.604996 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:41.605230 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:41.605231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:41.605232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:41.605348 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:41.605461 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:41.605584 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 18:48:41.605647 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 18:48:41.622937 4903 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:41.623718 4903 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:41.640432 4903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2bf68ef8cf54f5abb3e8d3ed2d27d05: Generated new TSK 0
I20260620 18:48:41.640667 4903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:41.650143 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 4823
W20260620 18:48:41.654713 4808 connection.cc:535] client connection to 127.15.77.126:44233 recv error: Network error: failed to read from TLS socket (remote: 127.15.77.126:44233): Connection reset by peer (error 104)
2026-06-20T18:48:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1357 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:41 dist-test-slave-tvl9 krb5kdc[4922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:41 dist-test-slave-tvl9 krb5kdc[4922](info): set up 2 sockets
Jun 20 18:48:41 dist-test-slave-tvl9 krb5kdc[4922](info): commencing operation
krb5kdc: starting...
W20260620 18:48:42.695487 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:42 dist-test-slave-tvl9 krb5kdc[4922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:42.817809 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:33381
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:42319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:33381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:42.899631 4935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:42.899875 4935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:42.899915 4935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:42.901081 4935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:42.901156 4935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:42.901180 4935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:42.901202 4935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:42.901230 4935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:42.902726 4935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:42319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:33381
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:33381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:42.903017 4935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:42.903257 4935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:42.908128 4941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:42.908174 4943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:42.908159 4940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:42.908355 4935 server_base.cc:1034] running on GCE node
I20260620 18:48:42.908622 4935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:42.908874 4935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:42.910025 4935 hybrid_clock.cc:648] HybridClock initialized: now 1781981322910010 us; error 31 us; skew 500 ppm
Jun 20 18:48:42 dist-test-slave-tvl9 krb5kdc[4922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981322, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:42.915258 4935 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:42.915786 4935 webserver.cc:460] Webserver started at http://127.15.77.126:46801/ using document root <none> and password file <none>
I20260620 18:48:42.915995 4935 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:42.916040 4935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:42.916133 4935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:42.917156 4935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "6bb423b124ba42f182238b835879def3"
format_stamp: "Formatted at 2026-06-20 18:48:42 on dist-test-slave-tvl9"
server_key: "9b16dbc9655cffeb0b201dbb29c64d89"
server_key_iv: "fcd7d92b2bc14457b3b4ddf44f296590"
server_key_version: "encryptionkey@0"
I20260620 18:48:42.917445 4935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "6bb423b124ba42f182238b835879def3"
format_stamp: "Formatted at 2026-06-20 18:48:42 on dist-test-slave-tvl9"
server_key: "9b16dbc9655cffeb0b201dbb29c64d89"
server_key_iv: "fcd7d92b2bc14457b3b4ddf44f296590"
server_key_version: "encryptionkey@0"
I20260620 18:48:42.918731 4935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:42.919492 4950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:42.919677 4935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:42.919782 4935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "6bb423b124ba42f182238b835879def3"
format_stamp: "Formatted at 2026-06-20 18:48:42 on dist-test-slave-tvl9"
server_key: "9b16dbc9655cffeb0b201dbb29c64d89"
server_key_iv: "fcd7d92b2bc14457b3b4ddf44f296590"
server_key_version: "encryptionkey@0"
I20260620 18:48:42.919867 4935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:42.941445 4935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:42.943410 4935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:42.943581 4935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:42.947686 4935 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:33381
I20260620 18:48:42.947726 5002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:33381 every 8 connection(s)
I20260620 18:48:42.948091 4935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:42.948971 5003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:42.951226 5003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3: Bootstrap starting.
I20260620 18:48:42.951838 5003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:42.952095 5003 log.cc:824] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:42.952757 5003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3: No bootstrap required, opened a new log
I20260620 18:48:42.952914 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 4935
I20260620 18:48:42.953019 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:42.953189 15669 external_mini_cluster.cc:1428] Setting key b13cf1e34f76d5c1210a379103ec67a3
I20260620 18:48:42.954779 5003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb423b124ba42f182238b835879def3" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33381 } }
I20260620 18:48:42.954945 5003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:42.954985 5003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb423b124ba42f182238b835879def3, State: Initialized, Role: FOLLOWER
I20260620 18:48:42.955183 5003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb423b124ba42f182238b835879def3" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33381 } }
I20260620 18:48:42.955291 5003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:42.955356 5003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:42.955407 5003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:42.955938 5003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb423b124ba42f182238b835879def3" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33381 } }
I20260620 18:48:42.956063 5003 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bb423b124ba42f182238b835879def3; no voters:
I20260620 18:48:42.956235 5003 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:42.956260 5008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:42.956425 5008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [term 1 LEADER]: Becoming Leader. State: Replica: 6bb423b124ba42f182238b835879def3, State: Running, Role: LEADER
I20260620 18:48:42.956489 5003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:42.956548 5008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb423b124ba42f182238b835879def3" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33381 } }
I20260620 18:48:42.956911 5010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb423b124ba42f182238b835879def3. Latest consensus state: current_term: 1 leader_uuid: "6bb423b124ba42f182238b835879def3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb423b124ba42f182238b835879def3" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33381 } } }
I20260620 18:48:42.957046 5010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:42.957306 5017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:42.957811 5017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:42.957930 5009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bb423b124ba42f182238b835879def3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb423b124ba42f182238b835879def3" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 33381 } } }
I20260620 18:48:42.958460 5009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:42.959944 5017 catalog_manager.cc:1347] Generated new cluster ID: 2cec9605574e41759968efa49e6b7f23
I20260620 18:48:42.960000 5017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:42.960276 5007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:42.953686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32793 (local address 127.15.77.126:33381)
0620 18:48:42.953871 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:42.953879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:42.953907 (+ 28us) server_negotiation.cc:407] Connection header received
0620 18:48:42.954759 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:42.954765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:42.954812 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:42.954924 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:42.955205 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:42.956685 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:42.957938 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:42.958618 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:42.958731 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:42.959547 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:42.959551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:42.959551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:42.959714 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:42.959746 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:42.959757 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:42.959864 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:42.960030 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 18:48:42.960075 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260620 18:48:42.961359 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:42.971027 5007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:42.967458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37495 (local address 127.15.77.126:33381)
0620 18:48:42.967521 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:42.967523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:42.967540 (+ 17us) server_negotiation.cc:407] Connection header received
0620 18:48:42.967563 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:42.967564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:42.967589 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:42.967658 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:42.967820 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:42.968587 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:42.969572 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:42.969925 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:42.969997 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:42.970762 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:42.970763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:42.970764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:42.970769 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:42.970785 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:42.970791 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:42.970854 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:42.970926 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 18:48:42.970940 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 18:48:42.975211 5017 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:42.975757 5017 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:42.991073 5017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bb423b124ba42f182238b835879def3: Generated new TSK 0
I20260620 18:48:42.991283 5017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:42.993844 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 4935
W20260620 18:48:42.998359 4920 connection.cc:535] client connection to 127.15.77.126:33381 recv error: Network error: failed to read from TLS socket (remote: 127.15.77.126:33381): Connection reset by peer (error 104)
2026-06-20T18:48:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:43 dist-test-slave-tvl9 krb5kdc[5034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:43 dist-test-slave-tvl9 krb5kdc[5034](info): set up 2 sockets
Jun 20 18:48:43 dist-test-slave-tvl9 krb5kdc[5034](info): commencing operation
krb5kdc: starting...
W20260620 18:48:44.031594 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:44 dist-test-slave-tvl9 krb5kdc[5034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:44.151029 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:43479
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:43479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:44.234395 5047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:44.234655 5047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:44.234696 5047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:44.235883 5047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:44.235956 5047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:44.235981 5047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:44.236011 5047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:44.236043 5047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:44.237524 5047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:43479
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:43479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:44.237809 5047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:44.238049 5047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:44.242880 5053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:44.242887 5055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:44.242887 5052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:44.243046 5047 server_base.cc:1034] running on GCE node
I20260620 18:48:44.243477 5047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:44.243741 5047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:44.244889 5047 hybrid_clock.cc:648] HybridClock initialized: now 1781981324244877 us; error 28 us; skew 500 ppm
Jun 20 18:48:44 dist-test-slave-tvl9 krb5kdc[5034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:44.247443 5047 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:44.247830 5047 webserver.cc:460] Webserver started at http://127.15.77.126:44485/ using document root <none> and password file <none>
I20260620 18:48:44.248068 5047 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:44.248111 5047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:44.248202 5047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:44.249194 5047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "e725a525658c4efc99259854655b7165"
format_stamp: "Formatted at 2026-06-20 18:48:44 on dist-test-slave-tvl9"
server_key: "a68b5336a9f13c501fb22c453b1a71fb"
server_key_iv: "6152a16cad722fcbeb8ed3b6298087a8"
server_key_version: "encryptionkey@0"
I20260620 18:48:44.249478 5047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "e725a525658c4efc99259854655b7165"
format_stamp: "Formatted at 2026-06-20 18:48:44 on dist-test-slave-tvl9"
server_key: "a68b5336a9f13c501fb22c453b1a71fb"
server_key_iv: "6152a16cad722fcbeb8ed3b6298087a8"
server_key_version: "encryptionkey@0"
I20260620 18:48:44.250720 5047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 18:48:44.251513 5062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:44.251681 5047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 18:48:44.251770 5047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "e725a525658c4efc99259854655b7165"
format_stamp: "Formatted at 2026-06-20 18:48:44 on dist-test-slave-tvl9"
server_key: "a68b5336a9f13c501fb22c453b1a71fb"
server_key_iv: "6152a16cad722fcbeb8ed3b6298087a8"
server_key_version: "encryptionkey@0"
I20260620 18:48:44.251827 5047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:44.255010 5047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:44.256955 5047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:44.257059 5047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:44.260908 5047 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:43479
I20260620 18:48:44.260977 5114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:43479 every 8 connection(s)
I20260620 18:48:44.261307 5047 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:44.262326 5115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:44.264883 5115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165: Bootstrap starting.
I20260620 18:48:44.265486 5115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:44.265787 5115 log.cc:824] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:44.265790 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 5047
I20260620 18:48:44.265949 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:44.266122 15669 external_mini_cluster.cc:1428] Setting key 8ca1791c83db167a3598066f11305bd1
I20260620 18:48:44.266628 5115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165: No bootstrap required, opened a new log
I20260620 18:48:44.268891 5115 raft_consensus.cc:348] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e725a525658c4efc99259854655b7165" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43479 } }
I20260620 18:48:44.269078 5115 raft_consensus.cc:374] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:44.269129 5115 raft_consensus.cc:729] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e725a525658c4efc99259854655b7165, State: Initialized, Role: FOLLOWER
I20260620 18:48:44.269450 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:44.266714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36213 (local address 127.15.77.126:43479)
0620 18:48:44.266933 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:44.266943 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:44.266976 (+ 33us) server_negotiation.cc:407] Connection header received
0620 18:48:44.267997 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:44.268007 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:44.268055 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:44.268211 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:44.268923 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:44.268927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:44.268927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:44.268932 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:44.268981 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:44.268994 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:44.269084 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:44.269245 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 18:48:44.269292 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260620 18:48:44.269284 5115 consensus_queue.cc:260] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e725a525658c4efc99259854655b7165" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43479 } }
I20260620 18:48:44.269629 5115 raft_consensus.cc:388] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:44.269690 5115 raft_consensus.cc:482] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:44.269742 5115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:44.270432 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:44.270447 5115 raft_consensus.cc:504] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e725a525658c4efc99259854655b7165" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43479 } }
I20260620 18:48:44.270551 5115 leader_election.cc:302] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e725a525658c4efc99259854655b7165; no voters:
I20260620 18:48:44.270706 5115 leader_election.cc:288] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:44.270763 5120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:44.270913 5120 raft_consensus.cc:686] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [term 1 LEADER]: Becoming Leader. State: Replica: e725a525658c4efc99259854655b7165, State: Running, Role: LEADER
I20260620 18:48:44.270938 5115 sys_catalog.cc:564] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:44.271052 5120 consensus_queue.cc:237] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e725a525658c4efc99259854655b7165" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43479 } }
I20260620 18:48:44.271441 5120 sys_catalog.cc:455] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e725a525658c4efc99259854655b7165. Latest consensus state: current_term: 1 leader_uuid: "e725a525658c4efc99259854655b7165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e725a525658c4efc99259854655b7165" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43479 } } }
I20260620 18:48:44.271680 5120 sys_catalog.cc:458] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:44.271586 5122 sys_catalog.cc:455] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e725a525658c4efc99259854655b7165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e725a525658c4efc99259854655b7165" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 43479 } } }
I20260620 18:48:44.272107 5122 sys_catalog.cc:458] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165 [sys.catalog]: This master's current role is: LEADER
W20260620 18:48:44.272943 5141 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:44.273017 5141 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:44.273138 5133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:44.273329 5133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:44.273722 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:44.272036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48441 (local address 127.15.77.126:43479)
0620 18:48:44.272085 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:44.272087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:44.272631 (+ 544us) server_negotiation.cc:407] Connection header received
0620 18:48:44.272744 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:44.272745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:44.272751 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:44.272807 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:44.273475 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:44.273477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:44.273478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:44.273482 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:44.273496 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:44.273500 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:44.273556 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:44.273620 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 18:48:44.273634 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 18:48:44.274806 5133 catalog_manager.cc:1347] Generated new cluster ID: d1e90cac2fbe464e93c475eb715ad1b9
I20260620 18:48:44.274861 5133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:44.296185 5133 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:44.296389 5133 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:44.306037 5133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e725a525658c4efc99259854655b7165: Generated new TSK 0
I20260620 18:48:44.306267 5133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:44.309919 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 5047
2026-06-20T18:48:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:44 dist-test-slave-tvl9 krb5kdc[5146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:44 dist-test-slave-tvl9 krb5kdc[5146](info): set up 2 sockets
Jun 20 18:48:44 dist-test-slave-tvl9 krb5kdc[5146](info): commencing operation
krb5kdc: starting...
W20260620 18:48:45.343415 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:45 dist-test-slave-tvl9 krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:45.467285 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:36635
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:42403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:45.548640 5159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:45.548859 5159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:45.548882 5159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:45.550083 5159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:45.550137 5159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:45.550153 5159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:45.550165 5159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:45.550177 5159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:45.551738 5159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:42403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:36635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:36635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:45.552016 5159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:45.552258 5159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:45.557317 5165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:45.557322 5164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:45.557335 5167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:45.557560 5159 server_base.cc:1034] running on GCE node
I20260620 18:48:45.557898 5159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:45.558203 5159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:45.559409 5159 hybrid_clock.cc:648] HybridClock initialized: now 1781981325559387 us; error 39 us; skew 500 ppm
Jun 20 18:48:45 dist-test-slave-tvl9 krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981325, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:45.562041 5159 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:45.562453 5159 webserver.cc:460] Webserver started at http://127.15.77.126:38551/ using document root <none> and password file <none>
I20260620 18:48:45.562654 5159 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:45.562693 5159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:45.562786 5159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:45.563905 5159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "c0e978c6158043b293bf21344308fdfe"
format_stamp: "Formatted at 2026-06-20 18:48:45 on dist-test-slave-tvl9"
server_key: "59a803a761482c9ce3233ad7302cb2c5"
server_key_iv: "847a2ce9fb4d36857dcc5e29f07e5b80"
server_key_version: "encryptionkey@0"
I20260620 18:48:45.564230 5159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "c0e978c6158043b293bf21344308fdfe"
format_stamp: "Formatted at 2026-06-20 18:48:45 on dist-test-slave-tvl9"
server_key: "59a803a761482c9ce3233ad7302cb2c5"
server_key_iv: "847a2ce9fb4d36857dcc5e29f07e5b80"
server_key_version: "encryptionkey@0"
I20260620 18:48:45.565523 5159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:48:45.566227 5174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:45.566404 5159 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:45.566517 5159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "c0e978c6158043b293bf21344308fdfe"
format_stamp: "Formatted at 2026-06-20 18:48:45 on dist-test-slave-tvl9"
server_key: "59a803a761482c9ce3233ad7302cb2c5"
server_key_iv: "847a2ce9fb4d36857dcc5e29f07e5b80"
server_key_version: "encryptionkey@0"
I20260620 18:48:45.566605 5159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:45.587945 5159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:45.589969 5159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:45.590166 5159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:45.594468 5159 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:36635
I20260620 18:48:45.594506 5226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:36635 every 8 connection(s)
I20260620 18:48:45.594882 5159 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:45.595909 5227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:45.598316 5227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe: Bootstrap starting.
I20260620 18:48:45.598969 5227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:45.599279 5227 log.cc:824] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:45.600082 5227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe: No bootstrap required, opened a new log
I20260620 18:48:45.601727 5227 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e978c6158043b293bf21344308fdfe" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36635 } }
I20260620 18:48:45.601871 5227 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:45.601894 5227 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e978c6158043b293bf21344308fdfe, State: Initialized, Role: FOLLOWER
I20260620 18:48:45.602063 5227 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e978c6158043b293bf21344308fdfe" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36635 } }
I20260620 18:48:45.602138 5227 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:45.602161 5227 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:45.602191 5227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:45.602373 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 5159
I20260620 18:48:45.602501 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:45.602676 15669 external_mini_cluster.cc:1428] Setting key 7382298d4b6206b6c90910fd1a0698ef
I20260620 18:48:45.603415 5227 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e978c6158043b293bf21344308fdfe" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36635 } }
I20260620 18:48:45.603554 5227 leader_election.cc:302] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0e978c6158043b293bf21344308fdfe; no voters:
I20260620 18:48:45.603724 5227 leader_election.cc:288] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:45.603747 5232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:45.603853 5232 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [term 1 LEADER]: Becoming Leader. State: Replica: c0e978c6158043b293bf21344308fdfe, State: Running, Role: LEADER
I20260620 18:48:45.603998 5227 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:45.603991 5232 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e978c6158043b293bf21344308fdfe" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36635 } }
I20260620 18:48:45.604496 5233 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0e978c6158043b293bf21344308fdfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e978c6158043b293bf21344308fdfe" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36635 } } }
I20260620 18:48:45.604604 5233 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:45.604832 5234 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0e978c6158043b293bf21344308fdfe. Latest consensus state: current_term: 1 leader_uuid: "c0e978c6158043b293bf21344308fdfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0e978c6158043b293bf21344308fdfe" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36635 } } }
I20260620 18:48:45.605199 5234 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:45.605638 5240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:45.606145 5240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 18:48:45 dist-test-slave-tvl9 krb5kdc[5146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:45.610837 5240 catalog_manager.cc:1347] Generated new cluster ID: 942ecac9e90c4d9c951b289bc1db5e6a
I20260620 18:48:45.610904 5240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:45.614382 5231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:45.603208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49891 (local address 127.15.77.126:36635)
0620 18:48:45.603415 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:45.603425 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:45.603462 (+ 37us) server_negotiation.cc:407] Connection header received
0620 18:48:45.603629 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:45.603637 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:45.603693 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:45.604665 (+ 972us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:45.604779 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:45.606186 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:45.607129 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:45.607761 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:45.607882 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:45.610430 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:45.610435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:45.610435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:45.610449 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:45.612705 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:45.613108 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:45.613113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:45.613114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:45.613249 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:45.613450 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:45.613451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:45.613451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:45.613786 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:45.613945 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:45.614025 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 18:48:45.614176 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":138,"threads_started":1}
I20260620 18:48:45.615422 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:45.621832 5240 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:45.622597 5240 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:45.624552 5231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:45.616870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38101 (local address 127.15.77.126:36635)
0620 18:48:45.616909 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:45.616910 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:45.617087 (+ 177us) server_negotiation.cc:407] Connection header received
0620 18:48:45.617147 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:45.617147 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:45.617157 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:45.617214 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:45.617858 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:45.618602 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:45.619680 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:45.619988 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:45.620055 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:45.621533 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:45.621535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:45.621535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:45.621542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:45.623340 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:45.623745 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:45.623746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:45.623747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:45.623804 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:45.624011 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:45.624012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:45.624012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:45.624134 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:45.624223 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:45.624368 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 18:48:45.624460 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:48:45.628331 5240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0e978c6158043b293bf21344308fdfe: Generated new TSK 0
I20260620 18:48:45.628547 5240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:45.632002 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 5159
2026-06-20T18:48:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:45 dist-test-slave-tvl9 krb5kdc[5258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:45 dist-test-slave-tvl9 krb5kdc[5258](info): set up 2 sockets
Jun 20 18:48:45 dist-test-slave-tvl9 krb5kdc[5258](info): commencing operation
krb5kdc: starting...
W20260620 18:48:46.663519 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:46 dist-test-slave-tvl9 krb5kdc[5258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:46.784145 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34291
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:40737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:34291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:46.867019 5271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:46.867230 5271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:46.867252 5271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:46.868460 5271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:46.868508 5271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:46.868523 5271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:46.868542 5271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:46.868554 5271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:46.869993 5271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:40737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:34291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:34291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:46.870434 5271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:46.870636 5271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:46.875648 5279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:46.875675 5277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:46.875648 5276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:46.875916 5271 server_base.cc:1034] running on GCE node
I20260620 18:48:46.876236 5271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:46.876552 5271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:46.877717 5271 hybrid_clock.cc:648] HybridClock initialized: now 1781981326877688 us; error 58 us; skew 500 ppm
Jun 20 18:48:46 dist-test-slave-tvl9 krb5kdc[5258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:46.880277 5271 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:46.880764 5271 webserver.cc:460] Webserver started at http://127.15.77.126:42595/ using document root <none> and password file <none>
I20260620 18:48:46.881014 5271 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:46.881086 5271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:46.881232 5271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:46.882292 5271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "152b27189f794c0c91b49ec397e3f3ac"
format_stamp: "Formatted at 2026-06-20 18:48:46 on dist-test-slave-tvl9"
server_key: "057233a674bad2163c2575925957b6d6"
server_key_iv: "b33253a9e525f7a436275792bf26aace"
server_key_version: "encryptionkey@0"
I20260620 18:48:46.882669 5271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "152b27189f794c0c91b49ec397e3f3ac"
format_stamp: "Formatted at 2026-06-20 18:48:46 on dist-test-slave-tvl9"
server_key: "057233a674bad2163c2575925957b6d6"
server_key_iv: "b33253a9e525f7a436275792bf26aace"
server_key_version: "encryptionkey@0"
I20260620 18:48:46.884083 5271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:46.884827 5286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:46.884995 5271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:46.885094 5271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "152b27189f794c0c91b49ec397e3f3ac"
format_stamp: "Formatted at 2026-06-20 18:48:46 on dist-test-slave-tvl9"
server_key: "057233a674bad2163c2575925957b6d6"
server_key_iv: "b33253a9e525f7a436275792bf26aace"
server_key_version: "encryptionkey@0"
I20260620 18:48:46.885178 5271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:46.894447 5271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:46.896474 5271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:46.896654 5271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:46.900873 5271 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:34291
I20260620 18:48:46.900907 5338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:34291 every 8 connection(s)
I20260620 18:48:46.901266 5271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:46.902137 5339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:46.904575 5339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac: Bootstrap starting.
I20260620 18:48:46.905208 5339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:46.905503 5339 log.cc:824] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:46.906195 5339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac: No bootstrap required, opened a new log
I20260620 18:48:46.907860 5339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152b27189f794c0c91b49ec397e3f3ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34291 } }
I20260620 18:48:46.908006 5339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:46.908054 5339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152b27189f794c0c91b49ec397e3f3ac, State: Initialized, Role: FOLLOWER
I20260620 18:48:46.908217 5339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152b27189f794c0c91b49ec397e3f3ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34291 } }
I20260620 18:48:46.908281 5339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:46.908303 5339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:46.908363 5339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:46.908968 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 5271
I20260620 18:48:46.909056 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:46.909327 15669 external_mini_cluster.cc:1428] Setting key 2f58198c5e90f83c160f5fb8737d9cfc
I20260620 18:48:46.909325 5339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152b27189f794c0c91b49ec397e3f3ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34291 } }
I20260620 18:48:46.909444 5339 leader_election.cc:302] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 152b27189f794c0c91b49ec397e3f3ac; no voters:
I20260620 18:48:46.909646 5339 leader_election.cc:288] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:46.909695 5342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:46.909919 5342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [term 1 LEADER]: Becoming Leader. State: Replica: 152b27189f794c0c91b49ec397e3f3ac, State: Running, Role: LEADER
I20260620 18:48:46.910003 5339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:46.910051 5342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152b27189f794c0c91b49ec397e3f3ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34291 } }
I20260620 18:48:46.910384 5342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 152b27189f794c0c91b49ec397e3f3ac. Latest consensus state: current_term: 1 leader_uuid: "152b27189f794c0c91b49ec397e3f3ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152b27189f794c0c91b49ec397e3f3ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34291 } } }
I20260620 18:48:46.910399 5344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "152b27189f794c0c91b49ec397e3f3ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152b27189f794c0c91b49ec397e3f3ac" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 34291 } } }
I20260620 18:48:46.910503 5342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:46.910712 5344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:46.910868 5352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:46.911693 5352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:46.913419 5352 catalog_manager.cc:1347] Generated new cluster ID: cd909cb6cda348948c8b17b64391d1cf
I20260620 18:48:46.913492 5352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:46.915745 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:46.909815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38825 (local address 127.15.77.126:34291)
0620 18:48:46.910332 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:46.910341 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:46.910373 (+ 32us) server_negotiation.cc:407] Connection header received
0620 18:48:46.910502 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:46.910510 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:46.910581 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:46.911016 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:46.911160 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:46.912528 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:46.913619 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:46.914213 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:46.914312 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:46.915052 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:46.915055 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:46.915056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:46.915255 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:46.915298 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:46.915332 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:46.915387 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:46.915484 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 18:48:46.915546 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":82,"threads_started":1}
I20260620 18:48:46.916791 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:46.921970 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:46.918384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47295 (local address 127.15.77.126:34291)
0620 18:48:46.918421 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:46.918422 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:46.918599 (+ 177us) server_negotiation.cc:407] Connection header received
0620 18:48:46.918685 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:46.918685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:46.918694 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:46.918740 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:46.918875 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:46.919642 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:46.920639 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:46.920925 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:46.921006 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:46.921721 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:46.921723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:46.921723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:46.921728 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:46.921742 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:46.921746 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:46.921794 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:46.921871 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 18:48:46.921896 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:48:46.932083 5352 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:46.932662 5352 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:46.939500 5352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 152b27189f794c0c91b49ec397e3f3ac: Generated new TSK 0
I20260620 18:48:46.939858 5352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:46.944651 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 5271
2026-06-20T18:48:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:46 dist-test-slave-tvl9 krb5kdc[5370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:47 dist-test-slave-tvl9 krb5kdc[5370](info): set up 2 sockets
Jun 20 18:48:47 dist-test-slave-tvl9 krb5kdc[5370](info): commencing operation
krb5kdc: starting...
W20260620 18:48:47.987663 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:48 dist-test-slave-tvl9 krb5kdc[5370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:48.106508 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:46133
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:36297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:46133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:48.188320 5383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:48.188542 5383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:48.188565 5383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:48.189757 5383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:48.189802 5383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:48.189816 5383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:48.189868 5383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:48.189888 5383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:48.191395 5383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:36297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:46133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:46133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:48.191660 5383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:48.191891 5383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 18:48:48.196882 5383 server_base.cc:1034] running on GCE node
W20260620 18:48:48.196856 5391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:48.196851 5389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:48.196849 5388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:48.197296 5383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:48.197641 5383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:48.198817 5383 hybrid_clock.cc:648] HybridClock initialized: now 1781981328198814 us; error 40 us; skew 500 ppm
Jun 20 18:48:48 dist-test-slave-tvl9 krb5kdc[5370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981328, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:48.204051 5383 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:48.204501 5383 webserver.cc:460] Webserver started at http://127.15.77.126:42811/ using document root <none> and password file <none>
I20260620 18:48:48.204734 5383 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:48.204802 5383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:48.204957 5383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:48.206060 5383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "1ff253a7ea21400884e68312836099b7"
format_stamp: "Formatted at 2026-06-20 18:48:48 on dist-test-slave-tvl9"
server_key: "7d99021ae1ee2a3ff9e91a59cec9bdfd"
server_key_iv: "4f6b869e74f1ae8e0e0ed97b2c624a8d"
server_key_version: "encryptionkey@0"
I20260620 18:48:48.206429 5383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "1ff253a7ea21400884e68312836099b7"
format_stamp: "Formatted at 2026-06-20 18:48:48 on dist-test-slave-tvl9"
server_key: "7d99021ae1ee2a3ff9e91a59cec9bdfd"
server_key_iv: "4f6b869e74f1ae8e0e0ed97b2c624a8d"
server_key_version: "encryptionkey@0"
I20260620 18:48:48.207849 5383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 18:48:48.208606 5398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:48.208775 5383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:48.208882 5383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "1ff253a7ea21400884e68312836099b7"
format_stamp: "Formatted at 2026-06-20 18:48:48 on dist-test-slave-tvl9"
server_key: "7d99021ae1ee2a3ff9e91a59cec9bdfd"
server_key_iv: "4f6b869e74f1ae8e0e0ed97b2c624a8d"
server_key_version: "encryptionkey@0"
I20260620 18:48:48.208956 5383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:48.212415 5383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:48.214737 5383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:48.214890 5383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:48.218823 5383 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:46133
I20260620 18:48:48.218870 5450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:46133 every 8 connection(s)
I20260620 18:48:48.219213 5383 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:48.220230 5451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:48.221099 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 5383
I20260620 18:48:48.221215 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:48.221391 15669 external_mini_cluster.cc:1428] Setting key 57b32830cbc40015d3c33073e4e397d7
I20260620 18:48:48.223742 5451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7: Bootstrap starting.
I20260620 18:48:48.223999 5454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:48.221932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41671 (local address 127.15.77.126:46133)
0620 18:48:48.222143 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:48.222152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:48.222185 (+ 33us) server_negotiation.cc:407] Connection header received
0620 18:48:48.222747 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:48.222757 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:48.222794 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:48.222946 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:48.223578 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:48.223581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:48.223582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:48.223586 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:48.223631 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:48.223645 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:48.223720 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:48.223774 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 18:48:48.223815 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260620 18:48:48.224478 5451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:48.224782 5451 log.cc:824] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:48.224910 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:48.225515 5451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7: No bootstrap required, opened a new log
I20260620 18:48:48.227468 5454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:48.226392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41859 (local address 127.15.77.126:46133)
0620 18:48:48.226432 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:48.226434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:48.226532 (+ 98us) server_negotiation.cc:407] Connection header received
0620 18:48:48.226597 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:48.226598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:48.226604 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:48.226654 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:48.227239 (+ 585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:48.227240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:48.227241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 18:48:48.227246 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:48.227259 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 18:48:48.227263 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:48.227331 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:48.227386 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 18:48:48.227400 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:48:48.227849 5451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff253a7ea21400884e68312836099b7" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46133 } }
I20260620 18:48:48.228027 5451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 18:48:48.228062 5451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ff253a7ea21400884e68312836099b7, State: Initialized, Role: FOLLOWER
I20260620 18:48:48.228180 5451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff253a7ea21400884e68312836099b7" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46133 } }
I20260620 18:48:48.228263 5451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 18:48:48.228286 5451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 18:48:48.228314 5451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:48.228835 5451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff253a7ea21400884e68312836099b7" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46133 } }
I20260620 18:48:48.228920 5451 leader_election.cc:302] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ff253a7ea21400884e68312836099b7; no voters:
I20260620 18:48:48.229048 5451 leader_election.cc:288] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 18:48:48.229100 5461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:48.229243 5461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [term 1 LEADER]: Becoming Leader. State: Replica: 1ff253a7ea21400884e68312836099b7, State: Running, Role: LEADER
I20260620 18:48:48.229285 5451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:48.229365 5461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff253a7ea21400884e68312836099b7" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46133 } }
I20260620 18:48:48.229714 5463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ff253a7ea21400884e68312836099b7. Latest consensus state: current_term: 1 leader_uuid: "1ff253a7ea21400884e68312836099b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff253a7ea21400884e68312836099b7" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46133 } } }
I20260620 18:48:48.229844 5463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:48.229708 5462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ff253a7ea21400884e68312836099b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff253a7ea21400884e68312836099b7" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 46133 } } }
I20260620 18:48:48.230026 5462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7 [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:48.230369 5469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:48.230846 5469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:48.232594 5469 catalog_manager.cc:1347] Generated new cluster ID: a9ad70072b75470f95f326209d48f289
I20260620 18:48:48.232664 5469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:48.249029 5469 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:48.249235 5469 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:48.253235 5469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ff253a7ea21400884e68312836099b7: Generated new TSK 0
I20260620 18:48:48.253445 5469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:48.256421 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 5383
2026-06-20T18:48:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1311 ms)
[----------] 18 tests from AuthTokenIssuingTest (23859 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:48 dist-test-slave-tvl9 krb5kdc[5482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:48 dist-test-slave-tvl9 krb5kdc[5482](info): set up 2 sockets
Jun 20 18:48:48 dist-test-slave-tvl9 krb5kdc[5482](info): commencing operation
krb5kdc: starting...
W20260620 18:48:49.299544 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:49.419885 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40177
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40177,127.15.77.125:35771,127.15.77.124:44369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:49.502043 5495 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 18:48:49.502257 5495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:49.502274 5495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:49.502295 5495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:49.503530 5495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:49.503573 5495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:49.503588 5495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:49.503619 5495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:49.503646 5495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:49.505224 5495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40177,127.15.77.125:35771,127.15.77.124:44369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:40177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:49.505491 5495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:49.505722 5495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:49.510614 5500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:49.510732 5495 server_base.cc:1034] running on GCE node
W20260620 18:48:49.510538 5503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:49.510538 5501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:49.511039 5495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:49.511379 5495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:49.512538 5495 hybrid_clock.cc:648] HybridClock initialized: now 1781981329512521 us; error 43 us; skew 500 ppm
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:49.514954 5495 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:49.515506 5495 webserver.cc:460] Webserver started at http://127.15.77.126:37739/ using document root <none> and password file <none>
I20260620 18:48:49.515743 5495 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:49.515810 5495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:49.515947 5495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:49.517022 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "e10ae5fa9c294093a869b0b6946de64c"
format_stamp: "Formatted at 2026-06-20 18:48:49 on dist-test-slave-tvl9"
server_key: "5e405d7e7751825a066d5a07f468915e"
server_key_iv: "7d0bd5ffb4e2703224565c86509d34b7"
server_key_version: "encryptionkey@0"
I20260620 18:48:49.517429 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "e10ae5fa9c294093a869b0b6946de64c"
format_stamp: "Formatted at 2026-06-20 18:48:49 on dist-test-slave-tvl9"
server_key: "5e405d7e7751825a066d5a07f468915e"
server_key_iv: "7d0bd5ffb4e2703224565c86509d34b7"
server_key_version: "encryptionkey@0"
I20260620 18:48:49.518756 5495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 18:48:49.519526 5510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:49.519706 5495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:49.519809 5495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "e10ae5fa9c294093a869b0b6946de64c"
format_stamp: "Formatted at 2026-06-20 18:48:49 on dist-test-slave-tvl9"
server_key: "5e405d7e7751825a066d5a07f468915e"
server_key_iv: "7d0bd5ffb4e2703224565c86509d34b7"
server_key_version: "encryptionkey@0"
I20260620 18:48:49.519896 5495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:49.531522 5495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:49.533341 5495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:49.533506 5495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:49.537830 5495 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:40177
I20260620 18:48:49.537868 5562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:40177 every 8 connection(s)
I20260620 18:48:49.538236 5495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:49.539180 5563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:49.540742 5563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:48:49.544906 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 5495
I20260620 18:48:49.545015 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:49.545192 15669 external_mini_cluster.cc:1428] Setting key 746a77545d7ba8702c47702dde42bb74
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:49.550529 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.541386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40177 (local address 127.0.0.1:35162)
0620 18:48:49.541620 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.541665 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:49.542161 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:49.542452 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:49.542455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:49.542728 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:49.542879 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.542884 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.543997 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.543999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.544681 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.544683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.545284 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.545286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.545350 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.545902 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:49.545910 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:49.547258 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:49.549052 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.549059 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.549061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.549259 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.549500 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.549501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.549502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.549588 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.549930 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:49.549935 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:49.549996 (+ 61us) client_negotiation.cc:769] Sending connection context
0620 18:48:49.550104 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 18:48:49.550336 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":112,"thread_start_us":95,"threads_started":1}
I20260620 18:48:49.550530 5566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.541454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35162 (local address 127.15.77.126:40177)
0620 18:48:49.541629 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.541638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.541767 (+ 129us) server_negotiation.cc:407] Connection header received
0620 18:48:49.542317 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.542319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.542338 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.542447 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.542992 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.543897 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.544781 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.545173 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.545287 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.547372 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.547379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.547379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.547385 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.548942 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.549349 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.549352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.549352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.549431 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.549654 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.549655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.549656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.549859 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.549950 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.550136 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.550291 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260620 18:48:49.551487 5563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:48:49.551869 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.551719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35771 (local address 127.0.0.1:40960)
0620 18:48:49.551778 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.551831 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 18:48:49.552024 5513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35771: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.15.77.125@KRBTEST.COM; defaulting to no policy
I20260620 18:48:49.552482 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.552355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35771 (local address 127.0.0.1:40976)
0620 18:48:49.552415 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.552448 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 18:48:49.552596 5563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.125:35771: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35771: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab.
Entry for principal kudu/127.15.77.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab.
I20260620 18:48:49.569942 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.125:35771
--webserver_interface=127.15.77.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab
--principal=kudu/127.15.77.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40177,127.15.77.125:35771,127.15.77.124:44369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:48:49.601854 5563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } attempt: 1
I20260620 18:48:49.602437 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.602279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35771 (local address 127.0.0.1:40990)
0620 18:48:49.602349 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.602399 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 18:48:49.602834 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.602737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35771 (local address 127.0.0.1:40994)
0620 18:48:49.602783 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.602813 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 18:48:49.602942 5563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.125:35771: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35771: connect: Connection refused (error 111)
W20260620 18:48:49.652578 5570 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 18:48:49.652818 5570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:49.652837 5570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:49.652858 5570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:49.654086 5570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:49.654125 5570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:49.654140 5570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:49.654194 5570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:49.654218 5570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:49.655759 5570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40177,127.15.77.125:35771,127.15.77.124:44369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab
--principal=kudu/127.15.77.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.125:35771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.77.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:49.655961 5570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:49.656134 5570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:49.661015 5576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:49.661019 5575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:49.661077 5578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:49.661242 5570 server_base.cc:1034] running on GCE node
I20260620 18:48:49.661479 5570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:49.661722 5570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:49.662870 5570 hybrid_clock.cc:648] HybridClock initialized: now 1781981329662846 us; error 38 us; skew 500 ppm
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:49.665400 5570 init.cc:377] Logged in from keytab as kudu/127.15.77.125@KRBTEST.COM (short username kudu)
I20260620 18:48:49.665786 5570 webserver.cc:460] Webserver started at http://127.15.77.125:44687/ using document root <none> and password file <none>
I20260620 18:48:49.665972 5570 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:49.666028 5570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:49.666117 5570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:49.667125 5570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/data/instance:
uuid: "21b3a5b28ef144caa806c470b6354ee4"
format_stamp: "Formatted at 2026-06-20 18:48:49 on dist-test-slave-tvl9"
server_key: "ccdb9b239368413c6b932af23ea81cf7"
server_key_iv: "bae1603085f7ce5d98402c8efac58944"
server_key_version: "encryptionkey@0"
I20260620 18:48:49.667500 5570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/wal/instance:
uuid: "21b3a5b28ef144caa806c470b6354ee4"
format_stamp: "Formatted at 2026-06-20 18:48:49 on dist-test-slave-tvl9"
server_key: "ccdb9b239368413c6b932af23ea81cf7"
server_key_iv: "bae1603085f7ce5d98402c8efac58944"
server_key_version: "encryptionkey@0"
I20260620 18:48:49.668735 5570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 18:48:49.669425 5585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:49.669616 5570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 18:48:49.669715 5570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/wal
uuid: "21b3a5b28ef144caa806c470b6354ee4"
format_stamp: "Formatted at 2026-06-20 18:48:49 on dist-test-slave-tvl9"
server_key: "ccdb9b239368413c6b932af23ea81cf7"
server_key_iv: "bae1603085f7ce5d98402c8efac58944"
server_key_version: "encryptionkey@0"
I20260620 18:48:49.669819 5570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:49.671070 5563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } attempt: 2
I20260620 18:48:49.671499 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.671380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35771 (local address 127.0.0.1:41004)
0620 18:48:49.671432 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.671469 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 18:48:49.671816 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.671723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35771 (local address 127.0.0.1:41016)
0620 18:48:49.671773 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.671795 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 18:48:49.671907 5563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.125:35771: Network error: Client connection negotiation failed: client connection to 127.15.77.125:35771: connect: Connection refused (error 111)
I20260620 18:48:49.687594 5570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:49.690053 5570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:49.690218 5570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:49.694325 5570 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.125:35771
I20260620 18:48:49.694361 5637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.125:35771 every 8 connection(s)
I20260620 18:48:49.694749 5570 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
I20260620 18:48:49.694913 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 5570
I20260620 18:48:49.695011 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-1/wal/instance
I20260620 18:48:49.695251 15669 external_mini_cluster.cc:1428] Setting key e6f1b109b9426b1641b900d8148236dd
I20260620 18:48:49.695906 5638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:49.697567 5638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.77.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.124@KRBTEST.COM" created.
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:49.710474 5566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.698339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35164 (local address 127.15.77.126:40177)
0620 18:48:49.698407 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.698410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.698627 (+ 217us) server_negotiation.cc:407] Connection header received
0620 18:48:49.699106 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.699107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.699124 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.699203 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.699948 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.700775 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.701788 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.702136 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.702221 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.707396 (+ 5175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.707398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.707398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.707406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.709006 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.709409 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.709410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.709411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.709477 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.709717 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.709718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.709718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.709878 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.709950 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.710217 (+ 267us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.710313 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 18:48:49.710650 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.698303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40177 (local address 127.0.0.1:35164)
0620 18:48:49.698493 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.698537 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:49.698999 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:49.699317 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:49.699322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:49.699649 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:49.699848 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.699853 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.700883 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.700883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.701697 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.701699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.702257 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.702259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.702385 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.703038 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:49.703051 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:49.707249 (+ 4198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:49.709092 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.709099 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.709101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.709332 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.709547 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.709548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.709548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.709653 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.709946 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:49.709951 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:49.710077 (+ 126us) client_negotiation.cc:769] Sending connection context
0620 18:48:49.710164 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 18:48:49.710367 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
I20260620 18:48:49.711289 5638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
I20260620 18:48:49.719882 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.711635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35771 (local address 127.0.0.1:41020)
0620 18:48:49.711697 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.711707 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:49.711748 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:49.712026 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:49.712027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:49.712180 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:49.712228 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.712230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.713087 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.713088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.713594 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.713595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.714089 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.714090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.714162 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.714663 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:49.714668 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:49.716998 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:49.718850 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.718851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.718852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.719021 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.719245 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.719245 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.719245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.719296 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.719643 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:49.719644 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:49.719684 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 18:48:49.719738 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 18:48:49.719812 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 18:48:49.720046 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.711710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41020 (local address 127.15.77.125:35771)
0620 18:48:49.711895 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.711899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.711916 (+ 17us) server_negotiation.cc:407] Connection header received
0620 18:48:49.711932 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.711933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.711943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.712026 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.712303 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.712996 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.713688 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.714003 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.714082 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.717080 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.717088 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.717089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.717095 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.718777 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.719093 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.719095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.719095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.719169 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.719436 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.719437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.719437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.719584 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.719650 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.719806 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.719910 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260620 18:48:49.720647 5638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:48:49.721083 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.720934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50092)
0620 18:48:49.720998 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.721065 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 18:48:49.721217 5589 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
I20260620 18:48:49.721649 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.721566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50102)
0620 18:48:49.721612 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.721635 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 18:48:49.721769 5638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:44369: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab.
Entry for principal kudu/127.15.77.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab.
I20260620 18:48:49.722652 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.124:44369
--webserver_interface=127.15.77.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab
--principal=kudu/127.15.77.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:40177,127.15.77.125:35771,127.15.77.124:44369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:48:49.763067 5563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } attempt: 3
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
I20260620 18:48:49.770531 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.763511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35771 (local address 127.0.0.1:41032)
0620 18:48:49.763574 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.763588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:49.763666 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:49.763818 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:49.763820 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:49.764003 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:49.764072 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.764074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.764748 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.764749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.765301 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.765303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.765728 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.765730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.765821 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.766356 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:49.766362 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:49.767821 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:49.769461 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.769462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.769463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.769643 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.769857 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.769857 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.769858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.769922 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.770179 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:49.770180 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:49.770230 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 18:48:49.770322 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 18:48:49.770425 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 18:48:49.770531 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.763595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.15.77.125:35771)
0620 18:48:49.763671 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.763673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.763702 (+ 29us) server_negotiation.cc:407] Connection header received
0620 18:48:49.763730 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.763731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.763746 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.763813 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.764164 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.764661 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.765395 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.765648 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.765730 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.767911 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.767913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.767913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.767922 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.769357 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.769727 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.769728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.769728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.769784 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.769992 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.769993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.769993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.770109 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.770187 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.770349 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.770427 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 18:48:49.770900 5563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:48:49.770998 5638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } attempt: 1
I20260620 18:48:49.771354 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.771206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50112)
0620 18:48:49.771248 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.771281 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260620 18:48:49.771370 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.771223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50114)
0620 18:48:49.771279 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.771334 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 18:48:49.771685 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.771599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50118)
0620 18:48:49.771633 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.771661 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260620 18:48:49.771787 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.771685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50120)
0620 18:48:49.771738 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.771765 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 18:48:49.771798 5563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:44369: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
W20260620 18:48:49.771867 5638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:44369: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
I20260620 18:48:49.802953 5563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } attempt: 1
I20260620 18:48:49.803458 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.803275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50126)
0620 18:48:49.803367 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.803416 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 18:48:49.803867 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.803748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50132)
0620 18:48:49.803803 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.803838 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 18:48:49.804001 5563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:44369: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
W20260620 18:48:49.807762 5645 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 18:48:49.807957 5645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:49.808002 5645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:49.808048 5645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:49.809347 5645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:49.809422 5645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:49.809439 5645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:49.809453 5645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:49.809464 5645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:49.811156 5645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:40177,127.15.77.125:35771,127.15.77.124:44369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab
--principal=kudu/127.15.77.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.124:44369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.77.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:49.811470 5645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:49.811719 5645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:49.816609 5650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:49.816619 5651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:49.816632 5653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:49.816694 5645 server_base.cc:1034] running on GCE node
I20260620 18:48:49.817123 5645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:49.817462 5645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:49.818629 5645 hybrid_clock.cc:648] HybridClock initialized: now 1781981329818623 us; error 35 us; skew 500 ppm
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:49.821249 5645 init.cc:377] Logged in from keytab as kudu/127.15.77.124@KRBTEST.COM (short username kudu)
I20260620 18:48:49.821666 5645 webserver.cc:460] Webserver started at http://127.15.77.124:41521/ using document root <none> and password file <none>
I20260620 18:48:49.821877 5645 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:49.821918 5645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:49.822118 5645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:49.823293 5645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/data/instance:
uuid: "c1de20d147984d4280314321967c0ce5"
format_stamp: "Formatted at 2026-06-20 18:48:49 on dist-test-slave-tvl9"
server_key: "77174f289da80f6b7833c448da810a72"
server_key_iv: "e5810a4696e7610c8fa9576122477957"
server_key_version: "encryptionkey@0"
I20260620 18:48:49.823712 5645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/wal/instance:
uuid: "c1de20d147984d4280314321967c0ce5"
format_stamp: "Formatted at 2026-06-20 18:48:49 on dist-test-slave-tvl9"
server_key: "77174f289da80f6b7833c448da810a72"
server_key_iv: "e5810a4696e7610c8fa9576122477957"
server_key_version: "encryptionkey@0"
I20260620 18:48:49.824985 5645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:49.825703 5660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:49.825881 5645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:49.825954 5645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/wal
uuid: "c1de20d147984d4280314321967c0ce5"
format_stamp: "Formatted at 2026-06-20 18:48:49 on dist-test-slave-tvl9"
server_key: "77174f289da80f6b7833c448da810a72"
server_key_iv: "e5810a4696e7610c8fa9576122477957"
server_key_version: "encryptionkey@0"
I20260620 18:48:49.826042 5645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:49.835836 5645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:49.838135 5645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:49.838294 5645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:49.840026 5638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } attempt: 2
I20260620 18:48:49.840480 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.840330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50146)
0620 18:48:49.840387 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.840436 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 18:48:49.840930 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.840834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50148)
0620 18:48:49.840867 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.840897 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260620 18:48:49.841048 5638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:44369: Network error: Client connection negotiation failed: client connection to 127.15.77.124:44369: connect: Connection refused (error 111)
I20260620 18:48:49.842502 5645 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.124:44369
I20260620 18:48:49.842546 5712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.124:44369 every 8 connection(s)
I20260620 18:48:49.842919 5645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
I20260620 18:48:49.843992 5713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:49.845556 5713 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:48:49.847417 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 5645
I20260620 18:48:49.847528 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781981032911509-15669-0/minicluster-data/master-2/wal/instance
I20260620 18:48:49.847733 15669 external_mini_cluster.cc:1428] Setting key 5d3d6502b78225415219ee62f0ab2058
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
I20260620 18:48:49.855613 5566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.846280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35170 (local address 127.15.77.126:40177)
0620 18:48:49.846344 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.846346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.846570 (+ 224us) server_negotiation.cc:407] Connection header received
0620 18:48:49.847018 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.847020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.847030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.847084 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.847860 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.848596 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.849871 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.850061 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.850165 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.852695 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.852697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.852697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.852707 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.854114 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.854565 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.854566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.854566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.854621 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.854899 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.854900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.854900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.855052 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.855109 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.855464 (+ 355us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.855548 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 18:48:49.855877 5715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.846213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:40177 (local address 127.0.0.1:35170)
0620 18:48:49.846432 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.846470 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:49.846936 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:49.847147 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:49.847150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:49.847513 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:49.847775 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.847781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.848688 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.848688 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.849773 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.849776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.850202 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.850203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.850306 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.850848 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:49.850855 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:49.852602 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:49.854188 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.854195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.854198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.854484 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.854689 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.854690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.854690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.854822 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.855123 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:49.855130 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:49.855294 (+ 164us) client_negotiation.cc:769] Sending connection context
0620 18:48:49.855444 (+ 150us) client_negotiation.cc:240] Negotiation successful
0620 18:48:49.855617 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260620 18:48:49.856593 5713 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:48:49.857352 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.848106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.15.77.124:44369)
0620 18:48:49.848252 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.848256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.848310 (+ 54us) server_negotiation.cc:407] Connection header received
0620 18:48:49.848368 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.848370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.848387 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.848443 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.849081 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.850054 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.850966 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.851429 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.851507 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.853996 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.854005 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.854005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.854016 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.855823 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.856227 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.856229 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.856230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.856298 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.856541 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.856542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.856543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.856700 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.856853 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.857095 (+ 242us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.857213 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:49.864076 5715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.856887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:35771 (local address 127.0.0.1:41044)
0620 18:48:49.856943 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.856955 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:49.856997 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:49.857126 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:49.857127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:49.857531 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:49.857566 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.857568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.858435 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.858437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.859120 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.859122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.859669 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.859670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.859736 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.860149 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:49.860154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:49.861449 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:49.862977 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.862978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.862978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.863197 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.863425 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.863426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.863426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.863500 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.863759 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:49.863760 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:49.863803 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 18:48:49.863888 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 18:48:49.863982 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 18:48:49.864085 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.856945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.15.77.125:35771)
0620 18:48:49.857009 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.857011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.857019 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:48:49.857047 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.857048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.857058 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.857114 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.857660 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.858362 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.859193 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.859551 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.859612 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.861534 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.861536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.861536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.861543 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.862874 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.863271 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.863272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.863273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.863365 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.863567 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.863568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.863569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.863684 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.863757 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.863916 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.864002 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 18:48:49.864815 5713 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:48:49.866344 5566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.858171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35174 (local address 127.15.77.126:40177)
0620 18:48:49.858212 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.858214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.858223 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:49.858244 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.858244 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.858253 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.858309 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.858859 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.859432 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.860464 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.860791 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.860858 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.863392 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.863394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.863394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.863401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.865040 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.865561 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.865562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.865562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.865612 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.865785 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.865786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.865787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.865908 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.866090 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.866169 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.866246 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
I20260620 18:48:49.872049 5715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.865102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50168)
0620 18:48:49.865138 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.865149 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:49.865183 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:49.865314 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:49.865315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:49.865452 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:49.865492 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.865494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.866219 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.866220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.866884 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.866886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.867399 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.867400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.867450 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.867847 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:49.867852 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:49.869225 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:49.870886 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.870887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.870888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.871116 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.871370 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.871370 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.871371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.871436 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.871705 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:49.871706 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:49.871740 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 18:48:49.871792 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 18:48:49.871913 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260620 18:48:49.872277 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.865134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50168 (local address 127.15.77.124:44369)
0620 18:48:49.865166 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.865168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.865175 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:48:49.865236 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.865237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.865245 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.865289 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.865678 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.866137 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.866957 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.867281 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.867399 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.869310 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.869335 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.869335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.869343 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.870804 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.871189 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.871190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.871191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.871250 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.871503 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.871505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.871505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.871638 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.871709 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.871842 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.871971 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":15}
I20260620 18:48:49.874308 5713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5: Bootstrap starting.
I20260620 18:48:49.875347 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.866986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41048 (local address 127.15.77.125:35771)
0620 18:48:49.867028 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.867030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.867065 (+ 35us) server_negotiation.cc:407] Connection header received
0620 18:48:49.867118 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.867118 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.867129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.867182 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.867715 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.868423 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.869359 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.869658 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.869744 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.872418 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.872419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.872420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.872427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.874148 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.874505 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.874506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.874507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.874566 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.874735 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.874736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.874737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.874897 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.875082 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.875119 (+ 37us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.875189 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:48:49.875411 5713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:49.875706 5713 log.cc:824] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:49.876592 5713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5: No bootstrap required, opened a new log
I20260620 18:48:49.877063 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:49.878811 5713 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } }
I20260620 18:48:49.878973 5713 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1de20d147984d4280314321967c0ce5, State: Initialized, Role: FOLLOWER
I20260620 18:48:49.879056 5713 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } }
I20260620 18:48:49.879175 5563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } attempt: 2
I20260620 18:48:49.878343 5537 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e10ae5fa9c294093a869b0b6946de64c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35174
I20260620 18:48:49.879470 5713 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:49.879423 5719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } } }
I20260620 18:48:49.879580 5719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 18:48:49.881366 5730 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:49.881421 5730 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:49.881634 5645 master_runner.cc:186] Error getting master registration for 127.15.77.126:40177: OK, instance_id { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" instance_seqno: 1781981329536637 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 18:48:49.883719 5645 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
I20260620 18:48:49.887233 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.879464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50178)
0620 18:48:49.879565 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.879577 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:49.879619 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:49.879841 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:49.879843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:49.880032 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:49.880099 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.880102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.881042 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.881043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.881768 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.881771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.882211 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.882212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.882271 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.882744 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:49.882751 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:49.884278 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:49.886096 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.886097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.886098 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.886320 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.886540 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.886541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.886542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.886612 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.886910 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:49.886911 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:49.886960 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 18:48:49.887039 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 18:48:49.887136 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260620 18:48:49.887236 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.879690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50178 (local address 127.15.77.124:44369)
0620 18:48:49.879725 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.879727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.879734 (+ 7us) server_negotiation.cc:407] Connection header received
0620 18:48:49.879751 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.879752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.879761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.879837 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.880201 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.880938 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.881846 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.882128 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.882204 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.884386 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.884388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.884389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.884395 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.886006 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.886412 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.886413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.886413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.886469 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.886681 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.886682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.886682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.886834 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.886898 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.887075 (+ 177us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.887148 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 18:48:49.888582 5563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c: Bootstrap starting.
I20260620 18:48:49.889221 5563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:49.889508 5563 log.cc:824] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:49.890386 5563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c: No bootstrap required, opened a new log
I20260620 18:48:49.891474 5563 raft_consensus.cc:348] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } }
I20260620 18:48:49.891606 5563 raft_consensus.cc:729] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10ae5fa9c294093a869b0b6946de64c, State: Initialized, Role: FOLLOWER
I20260620 18:48:49.891719 5563 consensus_queue.cc:260] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } }
I20260620 18:48:49.892040 5563 sys_catalog.cc:564] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:49.892071 5537 raft_consensus.cc:482] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 18:48:49.892139 5537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:49.892949 5537 raft_consensus.cc:504] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } }
I20260620 18:48:49.893230 5537 leader_election.cc:288] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [CANDIDATE]: Term 1 election: Requested vote from peers 21b3a5b28ef144caa806c470b6354ee4 (127.15.77.125:35771), c1de20d147984d4280314321967c0ce5 (127.15.77.124:44369)
I20260620 18:48:49.893656 5687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e10ae5fa9c294093a869b0b6946de64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c1de20d147984d4280314321967c0ce5"
I20260620 18:48:49.893822 5687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:49.894850 5687 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e10ae5fa9c294093a869b0b6946de64c in term 1.
I20260620 18:48:49.895409 5514 leader_election.cc:302] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1de20d147984d4280314321967c0ce5, e10ae5fa9c294093a869b0b6946de64c; no voters:
I20260620 18:48:49.893594 5733 sys_catalog.cc:455] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } } }
I20260620 18:48:49.895596 5733 sys_catalog.cc:458] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:49.895674 5733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:49.895725 5733 raft_consensus.cc:686] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [term 1 LEADER]: Becoming Leader. State: Replica: e10ae5fa9c294093a869b0b6946de64c, State: Running, Role: LEADER
I20260620 18:48:49.895807 5733 consensus_queue.cc:237] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } }
I20260620 18:48:49.896526 5752 sys_catalog.cc:455] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e10ae5fa9c294093a869b0b6946de64c. Latest consensus state: current_term: 1 leader_uuid: "e10ae5fa9c294093a869b0b6946de64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } } }
I20260620 18:48:49.896627 5752 sys_catalog.cc:458] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:49.897083 5754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:49.897632 5754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:49.897507 5495 master_runner.cc:186] Error getting master registration for 127.15.77.125:35771: OK, instance_id { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" instance_seqno: 1781981329693137 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 18:48:49.900058 5495 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:48:49.902601 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.894721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.15.77.125:35771)
0620 18:48:49.894770 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.894772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.894782 (+ 10us) server_negotiation.cc:407] Connection header received
0620 18:48:49.894880 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.894881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.894893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.894925 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.895809 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.896490 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.897527 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.897820 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.897863 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.899545 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.899546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.899547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.899554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.901350 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.901763 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.901765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.901765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.901822 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.902017 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.902017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.902018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.902150 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.902217 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.902418 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.902500 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 18:48:49.908543 5687 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [term 1 FOLLOWER]: Refusing update from remote peer e10ae5fa9c294093a869b0b6946de64c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:49.908877 5752 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:49.910055 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.895257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.15.77.124:44369)
0620 18:48:49.895293 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.895295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.899750 (+ 4455us) server_negotiation.cc:407] Connection header received
0620 18:48:49.899838 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.899839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.899849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.899879 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.902992 (+ 3113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.903794 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.904811 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.905106 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.905145 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.906565 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.906566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.906567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.906574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.908174 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.909182 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.909183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.909184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.909244 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.909453 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.909454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.909455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.909603 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.909674 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.909891 (+ 217us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.909968 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 18:48:49.910307 5719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e10ae5fa9c294093a869b0b6946de64c. Latest consensus state: current_term: 1 leader_uuid: "e10ae5fa9c294093a869b0b6946de64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } } }
I20260620 18:48:49.910403 5719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:49.910979 5566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.894926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.15.77.126:40177)
0620 18:48:49.895146 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.895148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.900118 (+ 4970us) server_negotiation.cc:407] Connection header received
0620 18:48:49.900142 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.900143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.900154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.900213 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.901740 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.902445 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.903795 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.904070 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.904115 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.905654 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.905656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.905656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.905663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.908603 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.909484 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.909486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.909486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.909548 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.909886 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.909887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.909887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.910030 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.910108 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.910799 (+ 691us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.910875 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260620 18:48:49.920513 5733 sys_catalog.cc:455] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e10ae5fa9c294093a869b0b6946de64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } } }
I20260620 18:48:49.920642 5733 sys_catalog.cc:458] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:49.920778 5756 mvcc.cc:205] Tried to move back new op lower bound from 7298995527302553600 to 7298995527254171648. Current Snapshot: MvccSnapshot[applied={T|T < 7298995527302553600}]
I20260620 18:48:49.921130 5719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e10ae5fa9c294093a869b0b6946de64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } } }
I20260620 18:48:49.921231 5719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:49.927182 5754 catalog_manager.cc:1347] Generated new cluster ID: 4dc81315133e4be79e6cfd87868a301f
I20260620 18:48:49.927254 5754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:49.932137 5638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } attempt: 3
I20260620 18:48:49.936582 5754 catalog_manager.cc:1370] Generated new certificate authority record
Jun 20 18:48:49 dist-test-slave-tvl9 krb5kdc[5482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981329, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
I20260620 18:48:49.938097 5754 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:49.940966 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.932440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:44369 (local address 127.0.0.1:50196)
0620 18:48:49.932532 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:49.932561 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:49.932606 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:49.932866 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:49.932867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:49.933059 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:49.933109 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.933111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.934247 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.934248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.935070 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:49.935072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.935687 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.935688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:49.935753 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.936267 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:49.936272 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:49.937907 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:49.939910 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.939911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.939911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.940128 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.940329 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:49.940329 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:49.940330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:49.940383 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:49.940650 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:49.940651 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:49.940697 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 18:48:49.940760 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 18:48:49.940857 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260620 18:48:49.941339 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:49.932686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50196 (local address 127.15.77.124:44369)
0620 18:48:49.932727 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.932729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.932737 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:48:49.932760 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.932761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.932771 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.932818 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.933191 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.933932 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.935171 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:49.935551 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:49.935623 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:49.938031 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:49.938033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:49.938033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:49.938040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:49.939816 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.940209 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.940211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.940211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.940264 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:49.940444 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:49.940444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:49.940445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:49.940566 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:49.940624 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:49.941153 (+ 529us) server_negotiation.cc:299] Negotiation successful
0620 18:48:49.941236 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:48:49.942896 5638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4: Bootstrap starting.
I20260620 18:48:49.943696 5638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:49.944012 5638 log.cc:824] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:49.945143 5638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4: No bootstrap required, opened a new log
I20260620 18:48:49.947155 5638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } }
I20260620 18:48:49.947296 5638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21b3a5b28ef144caa806c470b6354ee4, State: Initialized, Role: FOLLOWER
I20260620 18:48:49.947391 5638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } }
I20260620 18:48:49.947713 5638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:49.947777 5612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e10ae5fa9c294093a869b0b6946de64c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "21b3a5b28ef144caa806c470b6354ee4"
I20260620 18:48:49.947892 5612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:49.948729 5612 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e10ae5fa9c294093a869b0b6946de64c in term 1.
I20260620 18:48:49.948925 5611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [term 1 FOLLOWER]: Refusing update from remote peer e10ae5fa9c294093a869b0b6946de64c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:49.949846 5752 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [LEADER]: Connected to new peer: Peer: permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:49.951738 5762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e10ae5fa9c294093a869b0b6946de64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } } }
I20260620 18:48:49.951848 5762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:49.951905 5762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e10ae5fa9c294093a869b0b6946de64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } } }
I20260620 18:48:49.951964 5762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:49.952306 5777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e10ae5fa9c294093a869b0b6946de64c. Latest consensus state: current_term: 1 leader_uuid: "e10ae5fa9c294093a869b0b6946de64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } } }
I20260620 18:48:49.952435 5777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:49.953135 5773 catalog_manager.cc:1259] Loaded cluster ID: 4dc81315133e4be79e6cfd87868a301f
I20260620 18:48:49.953178 5773 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4: loading cluster ID for follower catalog manager: success
I20260620 18:48:49.953239 5754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c: Generated new TSK 0
I20260620 18:48:49.953409 5754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 18:48:49.954051 5773 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4: acquiring CA information for follower catalog manager: success
W20260620 18:48:49.954213 5773 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260620 18:48:49.972275 5733 sys_catalog.cc:455] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e10ae5fa9c294093a869b0b6946de64c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e10ae5fa9c294093a869b0b6946de64c" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 40177 } } peers { permanent_uuid: "21b3a5b28ef144caa806c470b6354ee4" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 35771 } } peers { permanent_uuid: "c1de20d147984d4280314321967c0ce5" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 44369 } } }
I20260620 18:48:49.972421 5733 sys_catalog.cc:458] T 00000000000000000000000000000000 P e10ae5fa9c294093a869b0b6946de64c [sys.catalog]: This master's current role is: LEADER
W20260620 18:48:49.985502 5566 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:49.983992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35192 (local address 127.15.77.126:40177)
0620 18:48:49.984033 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.984035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.984659 (+ 624us) server_negotiation.cc:407] Connection header received
0620 18:48:49.984866 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.984867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.984880 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.984910 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.985408 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35192: BlockingRecv error: recv got EOF from 127.0.0.1:35192 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 18:48:49.986212 5717 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:49.984405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50206 (local address 127.15.77.124:44369)
0620 18:48:49.984448 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.984450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.984812 (+ 362us) server_negotiation.cc:407] Connection header received
0620 18:48:49.985742 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.985744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.985757 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.985788 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.986151 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50206: BlockingRecv error: recv got EOF from 127.0.0.1:50206 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260620 18:48:49.987025 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:49.983750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.15.77.125:35771)
0620 18:48:49.983797 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.983799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.986468 (+ 2669us) server_negotiation.cc:407] Connection header received
0620 18:48:49.986625 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.986626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.986637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.986665 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.986962 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41064: BlockingRecv error: recv got EOF from 127.0.0.1:41064 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 18:48:49.989419 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:49.988723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.15.77.125:35771)
0620 18:48:49.988768 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.988769 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.988902 (+ 133us) server_negotiation.cc:407] Connection header received
0620 18:48:49.989010 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.989012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.989030 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.989060 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.989374 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41080: BlockingRecv error: recv got EOF from 127.0.0.1:41080 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260620 18:48:49.992547 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:49.991750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.15.77.125:35771)
0620 18:48:49.991786 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.991788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.992035 (+ 247us) server_negotiation.cc:407] Connection header received
0620 18:48:49.992131 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.992133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.992143 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.992172 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:49.992501 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41096: BlockingRecv error: recv got EOF from 127.0.0.1:41096 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 18:48:50.000109 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:49.995699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.15.77.125:35771)
0620 18:48:49.995733 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:49.995735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:49.999561 (+ 3826us) server_negotiation.cc:407] Connection header received
0620 18:48:49.999668 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:49.999669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:49.999681 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:49.999711 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:50.000059 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41110: BlockingRecv error: recv got EOF from 127.0.0.1:41110 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 18:48:50.006106 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:50.005310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.15.77.125:35771)
0620 18:48:50.005357 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:50.005359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:50.005554 (+ 195us) server_negotiation.cc:407] Connection header received
0620 18:48:50.005659 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:50.005660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:50.005672 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:50.005703 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:50.006055 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41114: BlockingRecv error: recv got EOF from 127.0.0.1:41114 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260620 18:48:50.016487 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:50.015510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41118 (local address 127.15.77.125:35771)
0620 18:48:50.015553 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:50.015555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:50.015866 (+ 311us) server_negotiation.cc:407] Connection header received
0620 18:48:50.015973 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:50.015974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:50.015986 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:50.016020 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:50.016392 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41118: BlockingRecv error: recv got EOF from 127.0.0.1:41118 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 18:48:50.034863 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:50.033885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41134 (local address 127.15.77.125:35771)
0620 18:48:50.033927 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:50.033930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:50.034239 (+ 309us) server_negotiation.cc:407] Connection header received
0620 18:48:50.034347 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:50.034348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:50.034362 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:50.034393 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:50.034801 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41134: BlockingRecv error: recv got EOF from 127.0.0.1:41134 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 18:48:50.069053 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:50.068232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.15.77.125:35771)
0620 18:48:50.068279 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:50.068282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:50.068423 (+ 141us) server_negotiation.cc:407] Connection header received
0620 18:48:50.068561 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:50.068562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:50.068581 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:50.068643 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:50.068992 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41148: BlockingRecv error: recv got EOF from 127.0.0.1:41148 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 18:48:50.134996 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:50.134143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.15.77.125:35771)
0620 18:48:50.134240 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:50.134243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:50.134392 (+ 149us) server_negotiation.cc:407] Connection header received
0620 18:48:50.134504 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:50.134505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:50.134517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:50.134571 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:50.134931 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41156: BlockingRecv error: recv got EOF from 127.0.0.1:41156 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260620 18:48:50.264978 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:50.264206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.15.77.125:35771)
0620 18:48:50.264285 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:50.264288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:50.264388 (+ 100us) server_negotiation.cc:407] Connection header received
0620 18:48:50.264490 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:50.264491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:50.264503 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:50.264565 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:50.264889 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41172: BlockingRecv error: recv got EOF from 127.0.0.1:41172 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260620 18:48:50.523083 5643 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:50.522286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41188 (local address 127.15.77.125:35771)
0620 18:48:50.522363 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:50.522365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:50.522445 (+ 80us) server_negotiation.cc:407] Connection header received
0620 18:48:50.522553 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:50.522554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:50.522566 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:50.522637 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:50.523017 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41188: BlockingRecv error: recv got EOF from 127.0.0.1:41188 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 18:48:50.881863 5730 catalog_manager.cc:1259] Loaded cluster ID: 4dc81315133e4be79e6cfd87868a301f
I20260620 18:48:50.881942 5730 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5: loading cluster ID for follower catalog manager: success
I20260620 18:48:50.882762 5730 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5: acquiring CA information for follower catalog manager: success
I20260620 18:48:50.883085 5730 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c1de20d147984d4280314321967c0ce5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:48:50.954862 5773 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 21b3a5b28ef144caa806c470b6354ee4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:48:51.038882 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:51.036346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41198 (local address 127.15.77.125:35771)
0620 18:48:51.036516 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:51.036518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:51.036548 (+ 30us) server_negotiation.cc:407] Connection header received
0620 18:48:51.036624 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:51.036625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:51.036676 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:51.036751 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:51.036934 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:51.037438 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:51.038202 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:51.038391 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:51.038444 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:51.038590 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:51.038697 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:51.038755 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:51.038795 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 18:48:51.038812 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":115,"threads_started":1}
I20260620 18:48:51.043193 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:51.040919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.15.77.124:44369)
0620 18:48:51.041020 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:51.041022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:51.041053 (+ 31us) server_negotiation.cc:407] Connection header received
0620 18:48:51.041098 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:51.041098 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:51.041134 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:51.041195 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:51.041292 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:51.041779 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:51.042471 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:51.042682 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:51.042739 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:51.042917 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:51.043017 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:51.043073 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:51.043113 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 18:48:51.043129 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":56,"threads_started":1}
I20260620 18:48:51.047957 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:51.044934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41214 (local address 127.15.77.125:35771)
0620 18:48:51.044975 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:51.044977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:51.045169 (+ 192us) server_negotiation.cc:407] Connection header received
0620 18:48:51.045274 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:51.045275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:51.045286 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:51.045337 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:51.045647 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:51.046367 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:51.047107 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:51.047388 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:51.047449 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:51.047636 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:51.047673 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:51.047702 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:51.047817 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 18:48:51.047866 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:48:51.048148 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:51.045112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50228 (local address 127.15.77.124:44369)
0620 18:48:51.045152 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:51.045153 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:51.045261 (+ 108us) server_negotiation.cc:407] Connection header received
0620 18:48:51.045342 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:51.045344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:51.045354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:51.045468 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:51.045563 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:51.046297 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:51.047298 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:51.047601 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:51.047674 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:51.047875 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:51.047907 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:51.047934 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:51.048036 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 18:48:51.048059 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 18:48:51.048237 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:51.044865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35208 (local address 127.15.77.126:40177)
0620 18:48:51.045024 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:51.045026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:51.045035 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:51.045162 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:51.045163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:51.045223 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:51.045300 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:51.045494 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:51.046062 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:51.047134 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:51.047469 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:51.047533 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:51.047704 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:51.047818 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:51.047945 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:51.048133 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 18:48:51.048154 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":56,"threads_started":1}
I20260620 18:48:51.050148 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 5495
I20260620 18:48:51.055744 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 5570
I20260620 18:48:51.061002 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 5645
2026-06-20T18:48:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2807 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 18:48:51 dist-test-slave-tvl9 krb5kdc[5863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 18:48:51 dist-test-slave-tvl9 krb5kdc[5863](info): set up 2 sockets
Jun 20 18:48:51 dist-test-slave-tvl9 krb5kdc[5863](info): commencing operation
krb5kdc: starting...
W20260620 18:48:52.095628 15669 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/test-user.keytab.
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T18:48:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T18:48:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.77.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
Entry for principal kudu/127.15.77.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab.
I20260620 18:48:52.214111 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:36973
--webserver_interface=127.15.77.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:36973,127.15.77.125:45439,127.15.77.124:42801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 18:48:52.294214 5876 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 18:48:52.294431 5876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:52.294449 5876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:52.294469 5876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:52.295559 5876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:52.295594 5876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:52.295608 5876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:52.295621 5876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:52.295634 5876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:52.297071 5876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:36973,127.15.77.125:45439,127.15.77.124:42801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.126.keytab
--principal=kudu/127.15.77.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.126:36973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.77.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:52.297273 5876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:52.297443 5876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:52.302460 5884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:52.302460 5882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:52.302500 5876 server_base.cc:1034] running on GCE node
W20260620 18:48:52.302446 5881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:52.302886 5876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:52.303143 5876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:52.304286 5876 hybrid_clock.cc:648] HybridClock initialized: now 1781981332304261 us; error 42 us; skew 500 ppm
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:52.306715 5876 init.cc:377] Logged in from keytab as kudu/127.15.77.126@KRBTEST.COM (short username kudu)
I20260620 18:48:52.307216 5876 webserver.cc:460] Webserver started at http://127.15.77.126:45671/ using document root <none> and password file <none>
I20260620 18:48:52.307487 5876 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:52.307533 5876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:52.307708 5876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:52.308756 5876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/data/instance:
uuid: "3db8b494a52942e38754fde3a7b8594e"
format_stamp: "Formatted at 2026-06-20 18:48:52 on dist-test-slave-tvl9"
server_key: "b66eca68ddbff5f10e204595dd96674e"
server_key_iv: "639cc53f720aa3ea5140fd78bccbeaf4"
server_key_version: "encryptionkey@0"
I20260620 18:48:52.309124 5876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/wal/instance:
uuid: "3db8b494a52942e38754fde3a7b8594e"
format_stamp: "Formatted at 2026-06-20 18:48:52 on dist-test-slave-tvl9"
server_key: "b66eca68ddbff5f10e204595dd96674e"
server_key_iv: "639cc53f720aa3ea5140fd78bccbeaf4"
server_key_version: "encryptionkey@0"
I20260620 18:48:52.310456 5876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 18:48:52.311148 5891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:52.311362 5876 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:52.311468 5876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/wal
uuid: "3db8b494a52942e38754fde3a7b8594e"
format_stamp: "Formatted at 2026-06-20 18:48:52 on dist-test-slave-tvl9"
server_key: "b66eca68ddbff5f10e204595dd96674e"
server_key_iv: "639cc53f720aa3ea5140fd78bccbeaf4"
server_key_version: "encryptionkey@0"
I20260620 18:48:52.311555 5876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:52.321233 5876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:52.323256 5876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:52.323468 5876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:52.327692 5876 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.126:36973
I20260620 18:48:52.327728 5943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.126:36973 every 8 connection(s)
I20260620 18:48:52.328097 5876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/data/info.pb
I20260620 18:48:52.328382 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 5876
I20260620 18:48:52.328488 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-0/wal/instance
I20260620 18:48:52.328727 15669 external_mini_cluster.cc:1428] Setting key 9c44e042f795dfdb240a6fbff7bc4d64
I20260620 18:48:52.329500 5944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:52.331274 5944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.77.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.125@KRBTEST.COM" created.
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:52.344431 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.332163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36973 (local address 127.0.0.1:46674)
0620 18:48:52.332401 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.332439 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:52.332997 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:52.333336 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:52.333338 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:52.333616 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:52.333781 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.333785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.335436 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.335437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.336130 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.336132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.336886 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.336887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.336940 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.337546 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:52.337555 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:52.340731 (+ 3176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:52.342797 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.342804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.342805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.343012 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.343284 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.343284 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.343285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.343433 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.343887 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:52.343892 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:52.343948 (+ 56us) client_negotiation.cc:769] Sending connection context
0620 18:48:52.344023 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 18:48:52.344154 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":108,"threads_started":1}
I20260620 18:48:52.344439 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.332233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46674 (local address 127.15.77.126:36973)
0620 18:48:52.332423 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.332434 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.332530 (+ 96us) server_negotiation.cc:407] Connection header received
0620 18:48:52.333177 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.333179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.333204 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.333322 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.333888 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.335295 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.336223 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.336778 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.336895 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.340819 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.340827 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.340827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.340834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.342718 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.343096 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.343100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.343101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.343210 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.343504 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.343505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.343505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.343816 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.343906 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.344090 (+ 184us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.344247 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260620 18:48:52.345392 5944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:48:52.345785 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.345659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:45439 (local address 127.0.0.1:37522)
0620 18:48:52.345710 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.345755 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:45439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 18:48:52.345948 5894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.125:45439: connect: Connection refused (error 111)
I20260620 18:48:52.346400 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.346309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:45439 (local address 127.0.0.1:37530)
0620 18:48:52.346359 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.346383 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:45439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 18:48:52.346560 5944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.125:45439: Network error: Client connection negotiation failed: client connection to 127.15.77.125:45439: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab.
Entry for principal kudu/127.15.77.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab.
I20260620 18:48:52.357932 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.125:45439
--webserver_interface=127.15.77.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab
--principal=kudu/127.15.77.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:36973,127.15.77.125:45439,127.15.77.124:42801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:48:52.395792 5944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } attempt: 1
I20260620 18:48:52.396340 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.396191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:45439 (local address 127.0.0.1:37542)
0620 18:48:52.396266 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.396309 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:45439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260620 18:48:52.396696 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.396593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:45439 (local address 127.0.0.1:37544)
0620 18:48:52.396640 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.396666 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:45439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 18:48:52.396783 5944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.125:45439: Network error: Client connection negotiation failed: client connection to 127.15.77.125:45439: connect: Connection refused (error 111)
W20260620 18:48:52.440302 5951 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 18:48:52.440529 5951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:52.440547 5951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:52.440568 5951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:52.441802 5951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:52.441864 5951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:52.441880 5951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:52.441892 5951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:52.441905 5951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:52.443524 5951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:36973,127.15.77.125:45439,127.15.77.124:42801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.125.keytab
--principal=kudu/127.15.77.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.125:45439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.77.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:52.443805 5951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:52.444053 5951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:52.449070 5957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:52.449056 5959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:52.449110 5951 server_base.cc:1034] running on GCE node
W20260620 18:48:52.449056 5956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:52.449486 5951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:52.449870 5951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:52.451047 5951 hybrid_clock.cc:648] HybridClock initialized: now 1781981332451032 us; error 44 us; skew 500 ppm
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:52.453812 5951 init.cc:377] Logged in from keytab as kudu/127.15.77.125@KRBTEST.COM (short username kudu)
I20260620 18:48:52.454281 5951 webserver.cc:460] Webserver started at http://127.15.77.125:40849/ using document root <none> and password file <none>
I20260620 18:48:52.454548 5951 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:52.454622 5951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:52.454775 5951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:52.455966 5951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/data/instance:
uuid: "3594c6f87bec46c9a8d943680e2800c8"
format_stamp: "Formatted at 2026-06-20 18:48:52 on dist-test-slave-tvl9"
server_key: "8c67848dcf910bea3678e2526d8b8619"
server_key_iv: "1fdb203b8a7515dea690213141cd79c5"
server_key_version: "encryptionkey@0"
I20260620 18:48:52.456342 5951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/wal/instance:
uuid: "3594c6f87bec46c9a8d943680e2800c8"
format_stamp: "Formatted at 2026-06-20 18:48:52 on dist-test-slave-tvl9"
server_key: "8c67848dcf910bea3678e2526d8b8619"
server_key_iv: "1fdb203b8a7515dea690213141cd79c5"
server_key_version: "encryptionkey@0"
I20260620 18:48:52.457731 5951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 18:48:52.458479 5966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:52.458680 5951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 18:48:52.458781 5951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/wal
uuid: "3594c6f87bec46c9a8d943680e2800c8"
format_stamp: "Formatted at 2026-06-20 18:48:52 on dist-test-slave-tvl9"
server_key: "8c67848dcf910bea3678e2526d8b8619"
server_key_iv: "1fdb203b8a7515dea690213141cd79c5"
server_key_version: "encryptionkey@0"
I20260620 18:48:52.458868 5951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:52.464916 5944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } attempt: 2
I20260620 18:48:52.465341 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.465213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:45439 (local address 127.0.0.1:37546)
0620 18:48:52.465275 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.465313 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:45439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 18:48:52.465718 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.465623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:45439 (local address 127.0.0.1:37554)
0620 18:48:52.465666 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.465690 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.125:45439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 18:48:52.465812 5944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.125:45439: Network error: Client connection negotiation failed: client connection to 127.15.77.125:45439: connect: Connection refused (error 111)
I20260620 18:48:52.476382 5951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:52.478312 5951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:52.478482 5951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:52.482771 5951 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.125:45439
I20260620 18:48:52.482825 6018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.125:45439 every 8 connection(s)
I20260620 18:48:52.483242 5951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/data/info.pb
I20260620 18:48:52.484411 6019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:52.486009 6019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:52.493489 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 5951
I20260620 18:48:52.493615 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-1/wal/instance
I20260620 18:48:52.493819 15669 external_mini_cluster.cc:1428] Setting key a64daea7e5bb21c01c52c87847a1ac33
I20260620 18:48:52.495039 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.486736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36973 (local address 127.0.0.1:46678)
0620 18:48:52.486899 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.486936 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:52.487490 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:52.487739 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:52.487742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:52.487995 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:52.488194 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.488198 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.488829 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.488830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.489609 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.489611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.490063 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.490064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.490170 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.490743 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:52.490752 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:52.492117 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:52.493641 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.493648 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.493650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.493936 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.494167 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.494168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.494168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.494267 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.494555 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:52.494559 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:52.494672 (+ 113us) client_negotiation.cc:769] Sending connection context
0620 18:48:52.494753 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 18:48:52.494870 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260620 18:48:52.495049 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.486761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.15.77.126:36973)
0620 18:48:52.486822 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.486824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.487046 (+ 222us) server_negotiation.cc:407] Connection header received
0620 18:48:52.487591 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.487592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.487607 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.487665 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.488271 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.488746 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.489686 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.489873 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.489926 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.492203 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.492204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.492205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.492211 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.493548 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.494016 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.494018 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.494018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.494099 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.494332 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.494333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.494334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.494489 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.494562 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.494798 (+ 236us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.494894 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 18:48:52.495675 6019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.77.124@KRBTEST.COM; defaulting to no policy
I20260620 18:48:52.503011 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.495947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:45439 (local address 127.0.0.1:37562)
0620 18:48:52.495998 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.496010 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:52.496053 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:52.496469 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:52.496470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:52.496616 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:52.496658 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.496660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.497499 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.497500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.497992 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.497993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.498484 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.498485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.498548 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.498955 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:52.498960 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:52.500113 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:52.501947 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.501948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.501948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.502114 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.502328 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.502329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.502329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.502383 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.502676 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:52.502677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:52.502716 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 18:48:52.502788 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 18:48:52.502886 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 18:48:52.503041 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.496057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37562 (local address 127.15.77.125:45439)
0620 18:48:52.496227 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.496230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.496246 (+ 16us) server_negotiation.cc:407] Connection header received
0620 18:48:52.496263 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.496264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.496274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.496467 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.496734 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.497395 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.498085 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.498381 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.498492 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.500230 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.500239 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.500239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.500246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.501857 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.502207 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.502209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.502210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.502265 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.502455 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.502456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.502456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.502606 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.502698 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.502819 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.502942 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260620 18:48:52.503804 6019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.77.124@KRBTEST.COM" created.
I20260620 18:48:52.504182 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.504062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58168)
0620 18:48:52.504100 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.504151 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260620 18:48:52.504305 5967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
I20260620 18:48:52.504748 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.504646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58180)
0620 18:48:52.504700 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.504725 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260620 18:48:52.504899 6019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:42801: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.77.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab.
Entry for principal kudu/127.15.77.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab.
I20260620 18:48:52.518525 15669 external_mini_cluster.cc:1326] Running /tmp/dist-test-task13qjC6/build/release/bin/kudu
/tmp/dist-test-task13qjC6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.124:42801
--webserver_interface=127.15.77.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab
--principal=kudu/127.15.77.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.77.84:34073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.77.126:36973,127.15.77.125:45439,127.15.77.124:42801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 18:48:52.554194 6019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } attempt: 1
I20260620 18:48:52.554828 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.554628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58188)
0620 18:48:52.554707 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.554779 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 18:48:52.555226 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.555119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58196)
0620 18:48:52.555171 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.555202 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 18:48:52.555372 6019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:42801: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
I20260620 18:48:52.556969 5944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } attempt: 3
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
I20260620 18:48:52.564411 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.557280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:45439 (local address 127.0.0.1:37568)
0620 18:48:52.557334 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.557347 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:52.557434 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:52.557575 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:52.557576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:52.557742 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:52.557806 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.557808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.558501 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.558502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.559038 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.559039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.559564 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.559565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.559650 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.560138 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:52.560143 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:52.561372 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:52.563471 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.563472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.563472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.563638 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.563849 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.563849 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.563849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.563909 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.564176 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:52.564177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:52.564217 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 18:48:52.564269 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 18:48:52.564341 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 18:48:52.564472 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.557359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.15.77.125:45439)
0620 18:48:52.557424 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.557426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.557438 (+ 12us) server_negotiation.cc:407] Connection header received
0620 18:48:52.557500 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.557501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.557513 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.557569 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.557910 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.558411 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.559125 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.559472 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.559562 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.561484 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.561486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.561486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.561495 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.563398 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.563716 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.563717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.563717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.563785 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.563967 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.563969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.563969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.564111 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.564176 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.564311 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.564388 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 18:48:52.564903 5944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } has no permanent_uuid. Determining permanent_uuid...
I20260620 18:48:52.565248 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.565149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58210)
0620 18:48:52.565198 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.565225 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 18:48:52.565578 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.565485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58222)
0620 18:48:52.565535 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.565559 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260620 18:48:52.565693 5944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:42801: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
I20260620 18:48:52.596861 5944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } attempt: 1
I20260620 18:48:52.597399 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.597257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58228)
0620 18:48:52.597328 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.597369 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260620 18:48:52.597846 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.597752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58234)
0620 18:48:52.597800 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.597827 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 18:48:52.597963 5944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:42801: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
W20260620 18:48:52.602486 6026 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 18:48:52.602684 6026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 18:48:52.602701 6026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 18:48:52.602722 6026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 18:48:52.603935 6026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 18:48:52.603973 6026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 18:48:52.603987 6026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 18:48:52.604000 6026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 18:48:52.604013 6026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 18:48:52.605412 6026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.77.84:34073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.77.126:36973,127.15.77.125:45439,127.15.77.124:42801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/krb5kdc/kudu_127.15.77.124.keytab
--principal=kudu/127.15.77.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.77.124:42801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.77.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 17:47:14 UTC on ccycloud.quasar-dtsakl.root.comops.site
build id 4457676
I20260620 18:48:52.605608 6026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 18:48:52.605791 6026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 18:48:52.610584 6031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:52.610577 6034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 18:48:52.610607 6032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 18:48:52.611061 6026 server_base.cc:1034] running on GCE node
I20260620 18:48:52.611271 6026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 18:48:52.611581 6026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 18:48:52.612725 6026 hybrid_clock.cc:648] HybridClock initialized: now 1781981332612710 us; error 28 us; skew 500 ppm
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 18:48:52.615231 6026 init.cc:377] Logged in from keytab as kudu/127.15.77.124@KRBTEST.COM (short username kudu)
I20260620 18:48:52.615720 6026 webserver.cc:460] Webserver started at http://127.15.77.124:39039/ using document root <none> and password file <none>
I20260620 18:48:52.615973 6026 fs_manager.cc:359] Metadata directory not provided
I20260620 18:48:52.616020 6026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 18:48:52.616179 6026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 18:48:52.617254 6026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/data/instance:
uuid: "7a590acbd95845a7a95a4a3e2f6b843a"
format_stamp: "Formatted at 2026-06-20 18:48:52 on dist-test-slave-tvl9"
server_key: "a6198033bed19c26657b75b1d900cdaa"
server_key_iv: "f73040e7afe7cb4d0425865d72d7dd81"
server_key_version: "encryptionkey@0"
I20260620 18:48:52.617628 6026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/wal/instance:
uuid: "7a590acbd95845a7a95a4a3e2f6b843a"
format_stamp: "Formatted at 2026-06-20 18:48:52 on dist-test-slave-tvl9"
server_key: "a6198033bed19c26657b75b1d900cdaa"
server_key_iv: "f73040e7afe7cb4d0425865d72d7dd81"
server_key_version: "encryptionkey@0"
I20260620 18:48:52.619419 6026 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260620 18:48:52.620213 6041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 18:48:52.620417 6026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 18:48:52.620487 6026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/data,/tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/wal
uuid: "7a590acbd95845a7a95a4a3e2f6b843a"
format_stamp: "Formatted at 2026-06-20 18:48:52 on dist-test-slave-tvl9"
server_key: "a6198033bed19c26657b75b1d900cdaa"
server_key_iv: "f73040e7afe7cb4d0425865d72d7dd81"
server_key_version: "encryptionkey@0"
I20260620 18:48:52.620584 6026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 18:48:52.623538 6019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } attempt: 2
I20260620 18:48:52.623993 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.623821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58250)
0620 18:48:52.623890 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.623938 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 18:48:52.624321 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.624220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58260)
0620 18:48:52.624266 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.624295 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 18:48:52.624418 6019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.77.124:42801: Network error: Client connection negotiation failed: client connection to 127.15.77.124:42801: connect: Connection refused (error 111)
I20260620 18:48:52.629459 6026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 18:48:52.631196 6026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 18:48:52.631302 6026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 18:48:52.635185 6026 rpc_server.cc:307] RPC server started. Bound to: 127.15.77.124:42801
I20260620 18:48:52.635231 6093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.77.124:42801 every 8 connection(s)
I20260620 18:48:52.635607 6026 server_base.cc:1166] Dumped server information to /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/data/info.pb
I20260620 18:48:52.636591 6094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 18:48:52.638168 6094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
I20260620 18:48:52.643769 15669 external_mini_cluster.cc:1388] Started /tmp/dist-test-task13qjC6/build/release/bin/kudu as pid 6026
I20260620 18:48:52.643895 15669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task13qjC6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781981032911509-15669-0/minicluster-data/master-2/wal/instance
I20260620 18:48:52.644107 15669 external_mini_cluster.cc:1428] Setting key 8c33aa1994fbb60c4f515f9bf32ae780
I20260620 18:48:52.647289 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.638885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46680 (local address 127.15.77.126:36973)
0620 18:48:52.638937 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.638939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.639138 (+ 199us) server_negotiation.cc:407] Connection header received
0620 18:48:52.639677 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.639678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.639688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.639748 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.640331 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.640804 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.641742 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.641922 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.641977 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.644301 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.644302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.644302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.644309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.645861 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.646318 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.646319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.646319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.646370 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.646618 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.646618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.646619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.646720 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.646803 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.647110 (+ 307us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.647194 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:48:52.647500 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.638837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.126:36973 (local address 127.0.0.1:46680)
0620 18:48:52.639005 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.639047 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:52.639588 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:52.639812 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:52.639815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:52.640059 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:52.640258 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.640262 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.640878 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.640879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.641663 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.641665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.642014 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.642015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.642110 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.642651 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:52.642660 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:52.644186 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:52.645942 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.645948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.645950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.646243 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.646432 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.646433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.646433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.646558 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.646809 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:52.646816 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:52.646961 (+ 145us) client_negotiation.cc:769] Sending connection context
0620 18:48:52.647074 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 18:48:52.647235 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260620 18:48:52.648069 6094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
I20260620 18:48:52.653620 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.644539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58268 (local address 127.15.77.124:42801)
0620 18:48:52.644690 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.644695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.644775 (+ 80us) server_negotiation.cc:407] Connection header received
0620 18:48:52.644838 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.644841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.644855 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.644923 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.645609 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.646609 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.647363 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.647850 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.647924 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.650734 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.650742 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.650742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.650749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.652354 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.652759 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.652763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.652764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.652821 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.653030 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.653031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.653031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.653179 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.653336 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.653419 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.653504 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260620 18:48:52.655596 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.648475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37578 (local address 127.15.77.125:45439)
0620 18:48:52.648546 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.648548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.648557 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:52.648584 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.648585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.648595 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.648666 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.648915 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.649663 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.650630 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.650921 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.651008 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.652818 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.652819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.652820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.652827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.654397 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.654833 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.654834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.654835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.654889 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.655134 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.655135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.655135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.655247 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.655298 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.655472 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.655533 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 18:48:52.655634 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.648324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.125:45439 (local address 127.0.0.1:37578)
0620 18:48:52.648374 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.648384 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:52.648420 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:52.648672 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:52.648673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:52.648787 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:52.648833 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.648834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.649740 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.649741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.650546 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.650549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.651018 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.651020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.651100 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.651523 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:52.651528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:52.652727 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:52.654543 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.654544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.654544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.654761 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.654968 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.654969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.654969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.655031 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.655309 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:52.655310 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:52.655375 (+ 65us) client_negotiation.cc:769] Sending connection context
0620 18:48:52.655452 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 18:48:52.655546 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 18:48:52.656137 6094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.126@KRBTEST.COM
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.124@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
I20260620 18:48:52.662494 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.654497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.15.77.126:36973)
0620 18:48:52.654550 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.654552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.654560 (+ 8us) server_negotiation.cc:407] Connection header received
0620 18:48:52.654583 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.654584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.654594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.654655 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.655081 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.655786 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.656640 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.656851 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.656937 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.659395 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.659398 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.659403 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.659411 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.661106 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.661466 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.661468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.661468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.661585 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.661873 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.661874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.661874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.662045 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.662220 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.662266 (+ 46us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.662350 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 18:48:52.662777 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.656398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58276)
0620 18:48:52.656425 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.656433 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:52.656497 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:52.656619 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:52.656620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:52.656747 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:52.656772 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.656773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.657388 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.657389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.657850 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.657851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.658241 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.658242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.658291 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.658702 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:52.658706 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:52.659999 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:52.661519 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.661519 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.661520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.661725 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.662039 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.662039 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.662040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.662118 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.662406 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:52.662407 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:52.662457 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 18:48:52.662546 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 18:48:52.662668 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260620 18:48:52.662823 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.656429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58276 (local address 127.15.77.124:42801)
0620 18:48:52.656477 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.656479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.656488 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:52.656554 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.656554 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.656565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.656608 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.656849 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.657297 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.657944 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.658160 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.658241 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.660084 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.660085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.660086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.660104 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.661442 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.661872 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.661874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.661874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.661940 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.662191 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.662192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.662192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.662327 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.662399 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.662584 (+ 185us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.662717 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":33}
I20260620 18:48:52.664527 6094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a: Bootstrap starting.
I20260620 18:48:52.665578 6094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:52.665939 6094 log.cc:824] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:52.666895 6094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a: No bootstrap required, opened a new log
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.77.125@KRBTEST.COM
I20260620 18:48:52.668751 6094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } }
I20260620 18:48:52.668911 6094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a590acbd95845a7a95a4a3e2f6b843a, State: Initialized, Role: FOLLOWER
I20260620 18:48:52.669008 6094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } }
I20260620 18:48:52.669397 6094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:52.669334 6100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } } }
I20260620 18:48:52.671288 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.663273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37584 (local address 127.15.77.125:45439)
0620 18:48:52.663336 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.663338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.663347 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:52.663398 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.663399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.663408 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.663471 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.663909 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.664601 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.665589 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.665845 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.665903 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.668424 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.668425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.668426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.668436 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.669977 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.670337 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.670338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.670338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.670402 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.670665 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.670666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.670666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.670810 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.670983 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.671085 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.671168 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 18:48:52.670799 6100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [sys.catalog]: This master's current role is: FOLLOWER
W20260620 18:48:52.671816 6111 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:52.671917 6111 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:52.673120 5944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } attempt: 2
I20260620 18:48:52.671872 6026 master_runner.cc:186] Error getting master registration for 127.15.77.126:36973: OK, instance_id { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" instance_seqno: 1781981332326444 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 18:48:52.674360 6026 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:48:52.675194 15669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 18:48:52.676429 5918 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3db8b494a52942e38754fde3a7b8594e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46696
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.126@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
I20260620 18:48:52.682232 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.673410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58280)
0620 18:48:52.674449 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.674463 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:52.674524 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:52.674678 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:52.674679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:52.674874 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:52.674928 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.674931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.675737 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.675738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.676476 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.676479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.676976 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.676977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.677038 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.677586 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:52.677591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:52.679148 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:52.681025 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.681026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.681026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.681276 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.681485 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.681486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.681486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.681560 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.681876 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:52.681877 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:52.681934 (+ 57us) client_negotiation.cc:769] Sending connection context
0620 18:48:52.682013 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 18:48:52.682130 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015}
I20260620 18:48:52.682252 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.673447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58280 (local address 127.15.77.124:42801)
0620 18:48:52.673490 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.673492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.674524 (+ 1032us) server_negotiation.cc:407] Connection header received
0620 18:48:52.674585 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.674586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.674595 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.674670 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.675003 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.675649 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.676570 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.676889 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.676965 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.679244 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.679246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.679246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.679254 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.680929 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.681357 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.681358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.681359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.681415 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.681627 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.681627 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.681628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.681803 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.681868 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.682049 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.682129 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 18:48:52.684080 5944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e: Bootstrap starting.
I20260620 18:48:52.684738 5944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:52.685047 5944 log.cc:824] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:52.685818 5944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e: No bootstrap required, opened a new log
I20260620 18:48:52.686841 5944 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } }
I20260620 18:48:52.686990 5944 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3db8b494a52942e38754fde3a7b8594e, State: Initialized, Role: FOLLOWER
I20260620 18:48:52.687114 5944 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } }
I20260620 18:48:52.687422 6114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } } }
I20260620 18:48:52.687485 5944 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:52.687515 6114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:52.687633 5918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 18:48:52.687743 5918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:52.688555 5918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } }
I20260620 18:48:52.689196 5918 leader_election.cc:288] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [CANDIDATE]: Term 1 election: Requested vote from peers 3594c6f87bec46c9a8d943680e2800c8 (127.15.77.125:45439), 7a590acbd95845a7a95a4a3e2f6b843a (127.15.77.124:42801)
W20260620 18:48:52.689391 6125 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:52.689450 6125 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:52.689478 6068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3db8b494a52942e38754fde3a7b8594e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7a590acbd95845a7a95a4a3e2f6b843a"
I20260620 18:48:52.689639 6068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:52.690501 6068 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3db8b494a52942e38754fde3a7b8594e in term 1.
I20260620 18:48:52.690801 5892 leader_election.cc:302] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3db8b494a52942e38754fde3a7b8594e, 7a590acbd95845a7a95a4a3e2f6b843a; no voters:
I20260620 18:48:52.690920 6114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 18:48:52.691059 6114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [term 1 LEADER]: Becoming Leader. State: Replica: 3db8b494a52942e38754fde3a7b8594e, State: Running, Role: LEADER
I20260620 18:48:52.691138 6114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } }
I20260620 18:48:52.691602 6114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3db8b494a52942e38754fde3a7b8594e. Latest consensus state: current_term: 1 leader_uuid: "3db8b494a52942e38754fde3a7b8594e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } } }
I20260620 18:48:52.691677 6114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:52.691906 6133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 18:48:52.692281 6133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 18:48:52.691874 5876 master_runner.cc:186] Error getting master registration for 127.15.77.125:45439: OK, instance_id { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" instance_seqno: 1781981332481557 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 18:48:52.694195 5876 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 18:48:52.695837 6068 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [term 1 FOLLOWER]: Refusing update from remote peer 3db8b494a52942e38754fde3a7b8594e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:52.696240 6127 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:52.697613 6100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3db8b494a52942e38754fde3a7b8594e. Latest consensus state: current_term: 1 leader_uuid: "3db8b494a52942e38754fde3a7b8594e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } } }
I20260620 18:48:52.697714 6100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:52.701134 6114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3db8b494a52942e38754fde3a7b8594e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } } }
I20260620 18:48:52.701223 6114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:52.701678 6100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3db8b494a52942e38754fde3a7b8594e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } } }
I20260620 18:48:52.701764 6100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:52.701941 6135 mvcc.cc:205] Tried to move back new op lower bound from 7298995538719023104 to 7298995538703622144. Current Snapshot: MvccSnapshot[applied={T|T < 7298995538719023104}]
I20260620 18:48:52.702829 6133 catalog_manager.cc:1347] Generated new cluster ID: b26838dd7bd84d479e2640a0a3e79cfd
I20260620 18:48:52.702883 6133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 18:48:52.712220 5948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.699594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46704 (local address 127.15.77.126:36973)
0620 18:48:52.699641 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.699643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.699950 (+ 307us) server_negotiation.cc:407] Connection header received
0620 18:48:52.700146 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.700147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.700159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.700190 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.701907 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.702686 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.704755 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.705067 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.705110 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.707968 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.707970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.707970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.707979 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.709872 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.710649 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.710651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.710652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.710719 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.711091 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.711092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.711093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.711238 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.711378 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.712029 (+ 651us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.712120 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 18:48:52.715677 6019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } attempt: 3
I20260620 18:48:52.719105 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.699648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.15.77.125:45439)
0620 18:48:52.699691 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.699692 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.707473 (+ 7781us) server_negotiation.cc:407] Connection header received
0620 18:48:52.707601 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.707602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.707616 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.707647 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.708303 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.709080 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.711046 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.711373 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.711429 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.714466 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.714467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.714468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.714476 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.716762 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.717377 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.717378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.717378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.717440 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.718487 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.718489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.718489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.718647 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.718718 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.718928 (+ 210us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.719007 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Jun 20 18:48:52 dist-test-slave-tvl9 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781981332, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.77.125@KRBTEST.COM for kudu/127.15.77.124@KRBTEST.COM
I20260620 18:48:52.723731 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.699946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58294 (local address 127.15.77.124:42801)
0620 18:48:52.699985 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.699987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.710219 (+ 10232us) server_negotiation.cc:407] Connection header received
0620 18:48:52.710319 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.710320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.710331 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.710364 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.711773 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.712469 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.713903 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.714253 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.714316 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.719863 (+ 5547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.719865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.719865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.719872 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.721832 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.722792 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.722793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.722793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.722853 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.723088 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.723089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.723089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.723221 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.723307 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.723531 (+ 224us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.723625 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 18:48:52.729599 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.715954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.77.124:42801 (local address 127.0.0.1:58296)
0620 18:48:52.716014 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 18:48:52.716026 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 18:48:52.716084 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 18:48:52.716583 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 18:48:52.716584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 18:48:52.716939 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0620 18:48:52.716982 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.716983 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.718646 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.718647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.719477 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 18:48:52.719480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.720779 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.720780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 18:48:52.720869 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.721928 (+ 1059us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 18:48:52.721933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 18:48:52.723554 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 18:48:52.728199 (+ 4645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.728200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.728201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.728432 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.728770 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 18:48:52.728771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 18:48:52.728771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 18:48:52.728830 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 18:48:52.729294 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 18:48:52.729295 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 18:48:52.729341 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 18:48:52.729388 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 18:48:52.729494 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 18:48:52.731083 6143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:52.716068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.15.77.124:42801)
0620 18:48:52.716329 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.716331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.716375 (+ 44us) server_negotiation.cc:407] Connection header received
0620 18:48:52.716399 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.716400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.716440 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.716664 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.717080 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.718508 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.719987 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:52.720625 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:52.721046 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:52.723668 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 18:48:52.723669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 18:48:52.723669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 18:48:52.723707 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 18:48:52.728032 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.728567 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.728569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.728569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.728659 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 18:48:52.728914 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 18:48:52.728914 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 18:48:52.728915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 18:48:52.729176 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 18:48:52.730122 (+ 946us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:52.730591 (+ 469us) server_negotiation.cc:299] Negotiation successful
0620 18:48:52.730826 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":110,"threads_started":1}
I20260620 18:48:52.733053 6019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8: Bootstrap starting.
I20260620 18:48:52.733892 6019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8: Neither blocks nor log segments found. Creating new log.
I20260620 18:48:52.734206 6019 log.cc:824] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8: Log is configured to *not* fsync() on all Append() calls
I20260620 18:48:52.735724 6019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8: No bootstrap required, opened a new log
I20260620 18:48:52.737767 6019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } }
I20260620 18:48:52.737933 6019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3594c6f87bec46c9a8d943680e2800c8, State: Initialized, Role: FOLLOWER
I20260620 18:48:52.738010 6019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } }
I20260620 18:48:52.738369 6019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 18:48:52.740168 5992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 18:48:52.741127 5992 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [term 1 FOLLOWER]: Refusing update from remote peer 3db8b494a52942e38754fde3a7b8594e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 18:48:52.741312 5993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3db8b494a52942e38754fde3a7b8594e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3594c6f87bec46c9a8d943680e2800c8"
I20260620 18:48:52.741460 6141 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 18:48:52.742139 5993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3db8b494a52942e38754fde3a7b8594e in term 1.
I20260620 18:48:52.743861 6159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3db8b494a52942e38754fde3a7b8594e. Latest consensus state: current_term: 1 leader_uuid: "3db8b494a52942e38754fde3a7b8594e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } } }
I20260620 18:48:52.744715 6151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3db8b494a52942e38754fde3a7b8594e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } } }
I20260620 18:48:52.744802 6151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:52.747153 6133 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 18:48:52.747958 6133 catalog_manager.cc:1504] Loading token signing keys...
I20260620 18:48:52.744030 6145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3db8b494a52942e38754fde3a7b8594e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } } }
I20260620 18:48:52.748171 6145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 18:48:52.752024 6161 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 18:48:52.752101 6161 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 18:48:52.744103 6159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 18:48:52.753129 6127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3db8b494a52942e38754fde3a7b8594e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3db8b494a52942e38754fde3a7b8594e" member_type: VOTER last_known_addr { host: "127.15.77.126" port: 36973 } } peers { permanent_uuid: "3594c6f87bec46c9a8d943680e2800c8" member_type: VOTER last_known_addr { host: "127.15.77.125" port: 45439 } } peers { permanent_uuid: "7a590acbd95845a7a95a4a3e2f6b843a" member_type: VOTER last_known_addr { host: "127.15.77.124" port: 42801 } } }
I20260620 18:48:52.753228 6127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e [sys.catalog]: This master's current role is: LEADER
I20260620 18:48:52.755944 6133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3db8b494a52942e38754fde3a7b8594e: Generated new TSK 0
I20260620 18:48:52.756196 6133 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 18:48:52.793105 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.791248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37600 (local address 127.15.77.125:45439)
0620 18:48:52.791299 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.791302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.792348 (+ 1046us) server_negotiation.cc:407] Connection header received
0620 18:48:52.792476 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.792478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.792490 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.792523 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.793025 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37600: BlockingRecv error: recv got EOF from 127.0.0.1:37600 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260620 18:48:52.793928 5948 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.791758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.15.77.126:36973)
0620 18:48:52.791801 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.791803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.793372 (+ 1569us) server_negotiation.cc:407] Connection header received
0620 18:48:52.793488 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.793489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.793502 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.793535 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.793861 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46706: BlockingRecv error: recv got EOF from 127.0.0.1:46706 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 18:48:52.794724 6143 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.792078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58302 (local address 127.15.77.124:42801)
0620 18:48:52.792120 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.792122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.794142 (+ 2020us) server_negotiation.cc:407] Connection header received
0620 18:48:52.794278 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.794279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.794294 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.794324 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.794662 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58302: BlockingRecv error: recv got EOF from 127.0.0.1:58302 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 18:48:52.797271 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.796010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.15.77.125:45439)
0620 18:48:52.796059 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.796061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.796712 (+ 651us) server_negotiation.cc:407] Connection header received
0620 18:48:52.796848 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.796849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.796871 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.796902 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.797220 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37606: BlockingRecv error: recv got EOF from 127.0.0.1:37606 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260620 18:48:52.800419 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.799581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.15.77.125:45439)
0620 18:48:52.799618 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.799619 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.799886 (+ 267us) server_negotiation.cc:407] Connection header received
0620 18:48:52.799987 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.799988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.800001 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.800030 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.800359 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37608: BlockingRecv error: recv got EOF from 127.0.0.1:37608 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 18:48:52.808537 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.803638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37622 (local address 127.15.77.125:45439)
0620 18:48:52.807533 (+ 3895us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.807535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.807925 (+ 390us) server_negotiation.cc:407] Connection header received
0620 18:48:52.808044 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.808046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.808059 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.808090 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.808478 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37622: BlockingRecv error: recv got EOF from 127.0.0.1:37622 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 18:48:52.814808 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.813855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37630 (local address 127.15.77.125:45439)
0620 18:48:52.813892 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.813894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.814207 (+ 313us) server_negotiation.cc:407] Connection header received
0620 18:48:52.814311 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.814312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.814323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.814354 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.814752 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37630: BlockingRecv error: recv got EOF from 127.0.0.1:37630 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 18:48:52.825210 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.824205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.15.77.125:45439)
0620 18:48:52.824259 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.824261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.824610 (+ 349us) server_negotiation.cc:407] Connection header received
0620 18:48:52.824721 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.824722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.824735 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.824765 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.825145 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37632: BlockingRecv error: recv got EOF from 127.0.0.1:37632 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260620 18:48:52.843612 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.842566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37640 (local address 127.15.77.125:45439)
0620 18:48:52.842613 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.842616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.842928 (+ 312us) server_negotiation.cc:407] Connection header received
0620 18:48:52.843040 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.843041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.843055 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.843086 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.843549 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37640: BlockingRecv error: recv got EOF from 127.0.0.1:37640 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 18:48:52.877763 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.876941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37652 (local address 127.15.77.125:45439)
0620 18:48:52.876993 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.876996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.877149 (+ 153us) server_negotiation.cc:407] Connection header received
0620 18:48:52.877247 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.877248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.877260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.877335 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.877689 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37652: BlockingRecv error: recv got EOF from 127.0.0.1:37652 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 18:48:52.943912 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:52.943024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.15.77.125:45439)
0620 18:48:52.943114 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:52.943116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:52.943260 (+ 144us) server_negotiation.cc:407] Connection header received
0620 18:48:52.943408 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:52.943409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:52.943428 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:52.943498 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:52.943861 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37666: BlockingRecv error: recv got EOF from 127.0.0.1:37666 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260620 18:48:53.074297 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:53.073467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.15.77.125:45439)
0620 18:48:53.073564 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:53.073567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:53.073656 (+ 89us) server_negotiation.cc:407] Connection header received
0620 18:48:53.073762 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:53.073764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:53.073776 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:53.073855 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:53.074225 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37678: BlockingRecv error: recv got EOF from 127.0.0.1:37678 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260620 18:48:53.332123 6023 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 18:48:53.331450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.15.77.125:45439)
0620 18:48:53.331504 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:53.331506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:53.331560 (+ 54us) server_negotiation.cc:407] Connection header received
0620 18:48:53.331624 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:53.331625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:53.331635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:53.331696 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 18:48:53.332044 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37686: BlockingRecv error: recv got EOF from 127.0.0.1:37686 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 18:48:53.672384 6111 catalog_manager.cc:1259] Loaded cluster ID: b26838dd7bd84d479e2640a0a3e79cfd
I20260620 18:48:53.672451 6111 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a: loading cluster ID for follower catalog manager: success
I20260620 18:48:53.673631 6111 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a: acquiring CA information for follower catalog manager: success
I20260620 18:48:53.673957 6111 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7a590acbd95845a7a95a4a3e2f6b843a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:48:53.752620 6161 catalog_manager.cc:1259] Loaded cluster ID: b26838dd7bd84d479e2640a0a3e79cfd
I20260620 18:48:53.752712 6161 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8: loading cluster ID for follower catalog manager: success
I20260620 18:48:53.753690 6161 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8: acquiring CA information for follower catalog manager: success
I20260620 18:48:53.754070 6161 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3594c6f87bec46c9a8d943680e2800c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 18:48:53.848182 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:53.845291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37698 (local address 127.15.77.125:45439)
0620 18:48:53.845434 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:53.845437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:53.845502 (+ 65us) server_negotiation.cc:407] Connection header received
0620 18:48:53.845555 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:53.845556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:53.845629 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:53.845723 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:53.845890 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:53.846405 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:53.847241 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:53.847487 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:53.847574 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:53.847771 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:53.847894 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:53.847986 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:53.848047 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 18:48:53.848075 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":55,"threads_started":1}
I20260620 18:48:53.852234 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:53.849858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58304 (local address 127.15.77.124:42801)
0620 18:48:53.849965 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:53.849966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:53.849975 (+ 9us) server_negotiation.cc:407] Connection header received
0620 18:48:53.850036 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:53.850036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:53.850074 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:53.850139 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:53.850249 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:53.850732 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:53.851506 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:53.851710 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:53.851765 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:53.851892 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:53.852000 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:53.852061 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:53.852094 (+ 33us) server_negotiation.cc:299] Negotiation successful
0620 18:48:53.852112 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260620 18:48:53.856822 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:53.854105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58312 (local address 127.15.77.124:42801)
0620 18:48:53.854148 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:53.854150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:53.854281 (+ 131us) server_negotiation.cc:407] Connection header received
0620 18:48:53.854360 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:53.854361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:53.854373 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:53.854422 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:53.854622 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:53.855301 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:53.856058 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:53.856346 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:53.856409 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:53.856550 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:53.856585 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:53.856637 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:53.856707 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 18:48:53.856730 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 18:48:53.857075 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:53.853919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.15.77.125:45439)
0620 18:48:53.853972 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:53.853975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:53.854168 (+ 193us) server_negotiation.cc:407] Connection header received
0620 18:48:53.854200 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:53.854202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:53.854215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:53.854282 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:53.854403 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:53.855189 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:53.856192 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:53.856488 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:53.856694 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:53.856718 (+ 24us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:53.856750 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:53.856939 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:53.856955 (+ 16us) server_negotiation.cc:299] Negotiation successful
0620 18:48:53.856993 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 18:48:53.857519 6241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 18:48:53.853872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.15.77.126:36973)
0620 18:48:53.853983 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 18:48:53.853985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 18:48:53.854148 (+ 163us) server_negotiation.cc:407] Connection header received
0620 18:48:53.854195 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 18:48:53.854196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 18:48:53.854257 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 18:48:53.854336 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 18:48:53.854537 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:53.855072 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:53.856461 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 18:48:53.856789 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 18:48:53.856851 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 18:48:53.857102 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 18:48:53.857218 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 18:48:53.857279 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 18:48:53.857383 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 18:48:53.857412 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
I20260620 18:48:53.863404 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 5876
I20260620 18:48:53.869469 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 5951
I20260620 18:48:53.874361 15669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task13qjC6/build/release/bin/kudu with pid 6026
2026-06-20T18:48:53Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2812 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5620 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (300968 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS